Image decoding device, image encoding device, and method thereof

ABSTRACT

A lossless decoding unit  52  takes quantization parameters of decoded blocks spatially or temporally adjacent to a block to be decoded, as selection candidates, and extracts, from stream information, difference information indicating difference as to a prediction quantization parameter selected from the selection candidates. A quantization parameter calculating unit  59  calculates, from the prediction quantization parameter and the difference information, a quantization parameter of the block to be decoded. Thus, decoding of the image can be performed correctly by calculating a quantization parameter equal to a quantization parameter used at the time of image encoding.

CROSS-REFERENCE TO PRIOR APPLICATION

This application is a continuation of U.S. patent application Ser. No.16/460,548 (filed on Jul. 2, 2019), which is a continuation of Ser. No.15/248,864 (filed on Aug. 26, 2016 and issued as U.S. Pat. No.10,419,761 on Sep. 17, 2019), which is a continuation of U.S. patentapplication Ser. No. 13/994,247 (filed on Jun. 14, 2013 and issued asU.S. Pat. No. 9,560,348 on Jan. 31, 2017), which is a National StagePatent Application of PCT International Patent Application No.PCT/JP2012/050456 (filed on Jan. 12, 2012) under 35 U.S.C. § 371, whichclaims priority to Japanese Patent Application Nos. 2011-153183 (filedon Jul. 11, 2011) and 2011-011861 (filed on Jan. 24, 2011), which areall hereby incorporated by reference in their entirety.

TECHNICAL FIELD

The present technology relates to an image decoding device, imageencoding device, and a method thereof. More particularly, encodingefficiency of quantization parameters is improved.

BACKGROUND ART

In recent years, there have come into widespread use devices whichhandle image information as digital in order to perform highly effectiveinformation transmission and storage at that time, for example,compliant to formats such as MPEG or the like to compress the image byorthogonal transform such as discrete cosine transform or the like andmotion compensation, both in broadcasting and general households.

In particular, MPEG2 (ISO/IEC 13818-2) is defined as a general-purposeimage encoding format, and has widely been employed now by a broad rangeof applications for professional usage and for consumer usage. Byemploying the MPEG2 compression format, a code amount (bit rate) of 4through 8 Mbps is allocated in the event of an interlaced scanning imageof standard resolution having 720×480 pixels, for example, whereby highcompression and good image quality can be realized. Also, a code amount(bit rate) of 18 through 22 Mbps is allocated in the event of aninterlaced scanning image of high resolution having 1920×1088 pixels,whereby high compression and good image quality can be realized.

Also, standardization has been performed as Joint Model ofEnhanced-Compression Video Coding which realizes higher encodingefficiency though greater computation amount is required for encodingand decoding thereof, and has become an international Standard calledH.264 and MPEG-4 Part 10 (hereinafter written as “H.264/AVC (AdvancedVideo Coding)”).

With this MPEG and J.264/AVC, at the time of quantizing macroblocks, thesize of quantization steps can be changed so that the compression rateis constant. Also, with MPEG, quantization parameters proportionate tothe quantization steps are used, and with H.264/AVC, quantizationparameters are used in which the parameter value increases by “6” whenthe quantization step doubles. In MPEG and H.264/AVC, quantizationparameters are encoded (see PTL 1).

CITATION LIST Patent Literature

PTL 1: Japanese Unexamined Patent Application Publication No.2006-094081

SUMMARY OF INVENTION Technical Problem

Now, with encoding processing of quantization parameters, in the eventthat the decoding order is in raster scan order as illustrated in, FIG.1 for example, a quantization parameter SliceQPY with an initial valueis used for the head macroblock of the slice. Subsequently, processingis performed in the decoding order indicated by the arrows, and thequantization parameters of this macroblock is updated by the differencevalue in quantization parameters as to the macroblock situated at theleft side (mb_qp_delta). Accordingly, there are cases where, when thedecoding order transitions from the block at the right edge to the blockat the left edge, the difference value becomes great since the image isdifferent, and encoding efficiency becomes poor. Also, encodingefficiency becomes poor in the event that the difference value as to themacroblock situated at the left side is great as well.

Further, with image compression technology, standardization is beingstudied for HEVC (High Efficiency Video Coding) which realizes evenhigher encoding efficiency than the H.264/AVC format. With this HDVC,basic units called coding units (CU: Coding Unit) which are extensionsof the concept of macroblocks. In the event that each block illustratedin FIG. 2 is a coding unit, the decoding order is the order of blockswith numbers sequentially increasing from “0”. In the event that thedecoding order is not raster scan order in this way, moving from block“7” to block “8”, for example, or from block “15” to block “16”, mayconceivably lead to a greater difference value since the spatialdistance is great.

Accordingly, it is an object of the present technology to improve theencoding efficiency of quantization parameters.

Solution to Problem

A first aspect of this technology is an image decoding device,including: an information acquiring unit configured to take quantizationparameters of decoded blocks spatially or temporally adjacent to a blockto be decoded, as selection candidates, and extract, from streaminformation, difference information indicating difference as to aprediction quantization parameter selected from the selectioncandidates; and a quantization parameter calculating unit configured tocalculate, from the prediction quantization parameter and the differenceinformation, a quantization parameter of the block to be decoded.

With this technology, difference information indicating difference as toa prediction quantization parameter selected from selection candidates,which are quantization parameters of decoded blocks spatially ortemporally adjacent to a block to be decoded, is extracted from streaminformation. Also, with the image decoding device, at least blocks wherequantization parameters are redundant or blocks where inversequantization using quantization parameters is not performed are excludedfrom quantization parameters of decoded blocks spatially or temporallyadjacent to the block to be decoded, and selection candidates are taken.For setting of the prediction quantization parameter, a quantizationparameter is selected in an order indicated by identificationinformation included in the stream information, with the adjacentdecoded blocks in a predetermined order, for example. Alternatively,determination is performed of selection candidates in an order setbeforehand, and the prediction quantization parameter is set based onthe determination result. Alternatively, one or the other is selected ofprocessing of setting to the prediction quantization parameter aquantization parameter in an order indicated by identificationinformation included in the stream information, and processing ofdetermining selection candidates in an order set beforehand and settingthe prediction quantization parameter, based on the determination resultincluded in the stream information. Further, with the image decodingdevice, a quantization parameter of the block to be decoded iscalculated by adding difference which the difference informationindicates to the prediction quantization parameter. Further, in theevent that there is no selection candidate, a quantization parameter ofan initial value in a slice is taken as the prediction quantizationparameter. Also, including a quantization parameter updated last in theselection candidates is also performed.

A second aspect of this technology is an image decoding method,including: a process of taking quantization parameters of decoded blocksspatially or temporally adjacent to a block to be decoded, as selectioncandidates, and extracting, from stream information, differenceinformation indicating difference as to a prediction quantizationparameter selected from the selection candidates; and a process ofcalculating, from the prediction quantization parameter and thedifference information, a quantization parameter of the block to bedecoded.

A third aspect of the present technology is an image encoding device,including: a control unit configured to set a quantization parameter asto a block to be encoded; an information generating unit configured totake quantization parameters of encoded blocks spatially or temporallyadjacent to a block to be encoded, as selection candidates, select fromthe selection candidates a prediction quantization parameter inaccordance to the set quantization parameter, and generate differenceinformation indicating difference between the prediction quantizationparameter and the set quantization parameters; and an encoding unitconfigured to include the difference information in stream informationgenerated by performing encoding processing of the block to be encoded,using the set quantization parameter.

With this technology, at least blocks where quantization parameters areredundant or blocks where quantization using quantization parameters isnot performed are excluded from quantization parameters of encodedblocks spatially or temporally adjacent to the block to be encoded, andselection candidates are taken. Also, a quantization parameter updatedlast or the like is also included in the selection candidates. Aquantization parameter of which the difference as to the quantizationparameter set from these selection candidates is the smallest, isselected as the prediction quantization parameter, and identificationinformation for selecting the prediction quantization parameter from theselection candidates is generated. For example, identificationinformation is the order of blocks corresponding to the selectedquantization parameter, with the adjacent encoded blocks in apredetermined order. Also, the predetermined array is an order of arraywhere priority is given to one of an encoded block adjacent to the leftside, an encoded block adjacent above, and an encoded block temporallyadjacent. Also, the order of array of adjacent encoded blocks can beswitched. Further, quantization parameters of encoded blocks temporallyadjacent may be reordered in accordance with parameter values, with theorder of selected quantization parameters being taken as identificationinformation. Also, determination of selection candidates may beperformed in an order set beforehand, with the prediction quantizationparameter being selected based on the determination result. Further,with the image encoding device, difference information indicating thedifference between the prediction quantization parameter and the setquantization parameter is generated. Also, in the event that there is noselection candidate, difference information indicating differencebetween a quantization parameter of an initial value in a slice, and theset quantization parameters, is generated. Also, selection can be madebetween processing of setting a quantization parameter of which thedifference as to the set quantization parameter is the smallest as theprediction quantization parameter, and processing of performingdetermination of selection candidates in an order set beforehand andselecting the prediction quantization parameter based on thedetermination result, and determination information indicating theselected processing is generated. The generated difference information,identification information, and determination information are includedin stream information generated by performing encoding processing of theblock to be encoded using the set quantization parameter.

A fourth aspect of this technology is an image encoding method,including: a process of setting a quantization parameter as to a blockto be encoded; a process of taking quantization parameters of encodedblocks spatially or temporally adjacent to a block to be encoded, asselection candidates, selecting from the selection candidates aprediction quantization parameter in accordance to the set quantizationparameter, and generating difference information indicating differencebetween the prediction quantization parameter and the set quantizationparameters; and a process of including the difference information instream information generated by performing encoding processing of theblock to be encoded, using the set quantization parameter.

Advantageous Effects of Invention

According to this technology, quantization parameters of encoded blocksspatially or temporally adjacent to a block to be encoded are taken asselection candidates, and a prediction quantization parameter isselected from the selection candidates in accordance with thequantization parameter set to the block to be encoded. Differenceinformation indicating difference between the prediction quantizationparameter and the quantization parameters set as to the block to beencoded is generated. Accordingly, the difference of quantizationparameters can be prevented from becoming a great value, and encodingefficiency of quantization parameters can be improved.

Also, in a case of decoding stream information where differenceinformation is included, a prediction quantization parameter is selectedfrom quantization parameters of decoded blocks spatially or temporallyadjacent to a block to be decoded, and a quantization parameter of theblock to be decoded is calculated from the prediction quantizationparameter and the difference information. Accordingly, even in the eventthat stream information is generated with improved encoding efficiencyof quantization parameters, the quantization parameters can be restoredbased on the prediction quantization parameter and differenceinformation when decoding this stream information, and decodingprocessing can be correctly performed.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating a case where decoding order is rasterscan order.

FIG. 2 is a diagram illustrating a case where decoding order is notraster scan order.

FIG. 3 is a diagram illustrating a configuration of an image encodingdevice.

FIG. 4 is a diagram illustrating a configuration of an informationgenerating unit.

FIG. 5 is a diagram exemplarily illustrating a hierarchical structure ofa coding unit.

FIG. 6 is a flowchart illustrating operations of an image encodingdevice.

FIG. 7 is a flowchart illustrating prediction processing.

FIG. 8 is a flowchart illustrating intra prediction processing.

FIG. 9 is a flowchart illustrating inter prediction processing.

FIG. 10 is a diagram for describing operations of an informationgenerating unit.

FIG. 11 is a diagram exemplarily illustrating operations of aninformation generating unit.

FIG. 12 is a flowchart illustrating processing regarding quantizationparameters in encoding.

FIG. 13 is a diagram exemplarily illustrating a sequence parameter set.

FIG. 14 is a flowchart illustrating frame encoding processing.

FIG. 15 is a diagram exemplarily illustrating a picture parameter set.

FIG. 16 is a diagram exemplarily illustrating a slice header.

FIG. 17 is a flowchart illustrating slice encoding processing.

FIG. 18 is a diagram illustrating the configuration of an image decodingdevice.

FIG. 19 is a diagram illustrating the configuration of quantizationparameter calculating unit.

FIG. 20 is a flowchart illustrating operations of an image decodingdevice.

FIG. 21 is a flowchart illustrating prediction image generatingprocessing.

FIG. 22 is a flowchart illustrating processing regarding quantizationparameters in decoding.

FIG. 23 is a flowchart for describing other operations of an imagedecoding device.

FIG. 24 is a diagram illustrating an operation example in a case ofimplicitly predicting quantization parameters.

FIG. 25 is a flowchart example of a case of implicitly predictingquantization parameters.

FIG. 26 illustrates another operation example in a case of implicitlypredicting quantization parameters.

FIG. 27 is a diagram exemplarily illustrating a program.

FIG. 28 is a flowchart for describing other operations of an imagedecoding device.

FIG. 29 is a diagram exemplifying a schematic configuration of acomputer device.

FIG. 30 is a diagram exemplifying a schematic configuration of atelevision receiver.

FIG. 31 is a diagram exemplifying a schematic configuration of acellular telephone.

FIG. 32 is a diagram exemplifying a schematic configuration of arecording/playback device.

FIG. 33 is a diagram exemplifying a schematic configuration of animaging apparatus.

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments of the present invention will be described.Note that description will proceed in the following order.

1. Configuration of Image Encoding Device 2. Operation of Image EncodingDevice

3. Generating Operation of Identification Information and DifferenceInformation based on Quantization Parameters

4. Configuration of Image Decoding Device 5. Operation of Image DecodingDevice 6. Other Operations of Image Encoding Device and Image DecodingDevice 7. Case of Software Processing 8. Case of Application toElectronic Equipment <1. Configuration of Image Encoding Device>

FIG. 3 illustrates a configuration of an image encoding device. Theimage encoding device 10 includes an Analog/Digital conversion unit (A/Dconversion unit) 11, a screen rearranging buffer 12, a subtracting unit13, an orthogonal transform unit 14, a quantization unit 15, a losslessencoding unit 16, a storage buffer 17, and a rate control unit 18.Further, the image encoding device 10 includes an inverse quantizationunit 21, an inverse orthogonal transform unit 22, an adding unit 23, adeblocking filter 24, a frame memory 26, a selector 26, an intraprediction unit 31, a motion prediction/compensation unit 32, and aprediction image/optimal mode selecting unit 33.

The A/D conversion unit 11 performs conversion of analog image signalsinto digital image data and outputs these to the screen rearrangingbuffer 12.

The screen rearranging buffer 12 performs rearranging of frames as tothe image data output from the A/D conversion unit 11. The screenrearranging buffer 12 performs rearranging of the frames according to aGOP (Group of Pictures) structure relating to the encoding processing,and outputs the image data after rearranging to the subtracting unit 13,rate control unit 18, intra prediction unit 31, and motionprediction/compensation unit 32.

The image data output from the screen rearranging buffer 12 and theprediction image data selected at the later-described predictionimage/optimal mode selecting unit 33 are supplied to the subtractingunit 13. The subtracting unit 13 calculates prediction error data whichis a difference between the image data output from the screenrearranging buffer 12 and the prediction image data supplied from theprediction image/optimal mode selecting unit 33, and outputs this to theorthogonal transform unit 14.

The orthogonal transform unit 14 performs orthogonal transformprocessing such as discrete cosine transform (DCT: Discrete CosineTransform), Karhunen-Loéve transform or the like, as to the predictionerror data output from the subtracting unit 13. The orthogonal transformunit 14 outputs the transform coefficient data obtained by performingorthogonal transform processing to the quantization unit 15.

The transform coefficient data output from the orthogonal transform unit14 and the quantization parameter (quantization scale) from alater-described information generating unit 19 are supplied to thequantization unit 15. The quantization unit 15 performs quantization ofthe transform coefficient data and outputs the quantized data to thelossless encoding unit 16 and inverse quantization unit 21. Also, thequantization unit 15 changes the bit rate of quantized data based on thequantization parameters set at the rate control unit 18.

The quantized data output from the quantization unit 15, identificationinformation and difference information from the later-describedinformation generating unit 19, prediction mode information from theintra prediction unit 31, and prediction mode information and differencemotion vector information and the like from the motionprediction/compensation unit 32, are supplied to the lossless encodingunit 16. Also, information indicating whether the optimal mode is intraprediction or inter prediction is supplied from the predictionimage/optimal mode selecting unit 33. Note that the prediction modeinformation includes prediction mode and block size information ofmotion prediction unit and so forth, according to whether intraprediction or inter prediction.

The lossless encoding unit 16 performs lossless encoding processing asto the quantized data, for example using variable length coding,arithmetic coding, or the like, to generate stream information andoutputs this to the storage buffer 17. Also, in the event the optimalmode is intra prediction, the lossless encoding unit 16 performslossless encoding on the prediction mode information supplied from theintra prediction unit 31. Also, in the event the optimal mode is interprediction, the lossless encoding unit 16 performs lossless encoding onthe prediction mode information and difference motion vectors and thelike supplied from the motion prediction/compensation unit 32. Further,the lossless encoding unit 16 performs lossless encoding of informationrelating to the quantization parameters, such as difference informationfor example. The lossless encoding unit 16 includes the informationfollowing lossless encoding in stream information.

The storage buffer 17 stores an encoded stream from the losslessencoding unit 16. Also, the storage buffer 17 outputs the stored encodedstream with a transmission speed in accordance with the transmissionpath.

The rate control unit 18 performs monitoring of a available capacity ofthe storage buffer 17, and sets the quantization parameters such that,in the event that there is little capacity available, the bit rate ofthe quantized data drops, and in the event that there is sufficientcapacity the bit rate of the quantized data rises. Also, the ratecontrol unit 18 detects complexity of the image, such as activity whichis information indicating the variance of pixel values for example,using image data supplied from the screen rearranging buffer 12. Therate control unit 18 the quantization parameters such that roughquantization is realized for image portions where the variance value ofpixels is low and fine quantization for portions otherwise, for example,based on the detection results of complexity of the image. The ratecontrol unit 18 outputs the quantization parameters that have been setto the information generating unit 19.

The information generating unit 19 outputs the quantization parameterssupplied from the rate control unit 18 to the quantization unit 15. Theinformation generating unit 19 also takes quantization parameters ofencoded blocks spatially or temporally adjacent to a block to be encodedas selection candidates. The information generating unit 19 selects aquantization parameter from the selection candidates in accordance withthe quantization parameter set at the rate control unit 18, and takes asa prediction quantization parameter. Further, the information generatingunit 19 generates identification information corresponding to theselected quantization parameters, i.e., identification information forselecting prediction quantization parameters from the selectioncandidates, and difference information indicating the difference betweenthe prediction quantization parameters and the set quantizationparameters.

FIG. 4 illustrates a configuration of the information generating unit.The information generating unit 19 has a quantization parameter memoryunit 191 and a difference computing unit 192. The information generatingunit 19 outputs to the quantization unit 15 the quantization parameterssupplied from the rate control unit 18. Also, the information generatingunit 19 supplies the quantization parameters supplied from the ratecontrol unit 18 to the quantization parameter memory unit 191 anddifference computing unit 192.

The quantization parameter memory unit 191 stores the suppliedquantization parameters. The difference computing unit 192 reads out,from the quantization parameters of encoded blocks stored in thequantization parameter memory unit 191, quantization parameters ofencoded blocks in the spatial or temporal periphery of the block to beencoded, as selection candidates. Also, at least blocks wherequantization parameters are redundant, and blocks where quantizationusing quantization parameters is not performed, such as blocks wheretransform coefficient data to be quantized at the quantization unit 15is all “0” for example, are excluded from selection candidates. Also,the difference computing unit 192 excludes from selection candidates,blocks (hereinafter referred to as “skip blocks”) regarding whichdetermination has been made to perform skip processing based oninformation from the later-described motion prediction/compensation unit32 and prediction image/optimal mode selecting unit 33.

The difference computing unit 192 selects a quantization parameter fromthe quantization parameters, in accordance with the quantizationparameter of the block to be encoded, i.e., the quantization parametersupplied from the rate control unit 18, as a prediction quantizationparameter. The difference computing unit 192 further generatesdifference information indicating the difference between identificationinformation for selecting prediction quantization parameters from theselection candidates and a quantization parameter of the block to beencoded, and outputs this to the lossless encoding unit 16.

Returning to FIG. 3, the inverse quantization unit 21 performs inversequantization processing of the quantized data supplied from thequantization unit 15. The inverse quantization unit 21 outputs thetransform coefficient data obtained by performing inverse quantizationprocessing to the inverse orthogonal transform unit 22.

The inverse orthogonal transform unit 22 performs inverse transformprocessing of the transform coefficient data supplied from the inversequantization unit 21, and outputs the obtained data to the adding unit23.

The adding unit 23 adds data supplied from the inverse orthogonaltransform unit 22 and prediction image data supplied from the predictionimage/optimal mode selecting unit 33 to generate decoded image data, andoutputs to the deblocking filter 24 and frame memory 26. Note that thedecoded image data is used as image data of the reference image.

The deblocking filter 24 performs filtering processing to decrease blockdistortion which occurs at the time of image encoding. The deblockingfilter 24 performs filtering processing to remove the block distortionfrom the decoded image data supplied from the adding unit 23, andoutputs the decoded image data after filtering processing in the framememory 26.

The frame memory 26 holds the decoded image data after filteringprocessing supplied from the deblocking filter 24. The decoded imageheld in the frame memory 26 is supplied to the motionprediction/compensation unit 32 as reference image data.

The intra prediction unit 31 performs prediction in intra predictionprocessing of all the candidate intra prediction modes, using inputimage data of the image to be encoded supplied from the screenrearranging buffer 12 and reference image data supplied from the addingunit 23, and determines an optimal intra prediction mode. The intraprediction unit 31 calculates a cost function value for each intraprediction mode for example, and takes the intra prediction mode wherethe encoding efficiency is best, based on the calculated cost functionvalue, as the optimal intra prediction mode. The intra prediction unit31 outputs the prediction image data generated in the optimal intraprediction mode, and the cost function value of the optimal intraprediction mode, to the prediction image/optimal mode selecting unit 33.Further, the intra prediction unit 31 outputs prediction modeinformation indicating the intra prediction mode to the losslessencoding unit 16.

The motion prediction/compensation unit 32 performs prediction in allcandidate inter prediction modes, using the input image data of theimage to be encoded, supplied from the screen rearranging buffer 12, andthe reference image data supplied from the frame memory 26, and decidesthe optimal inter prediction mode. The motion prediction/compensationunit 32 calculates a cost function value in each inter prediction modefor example, and takes the inter prediction mode where the encodingefficiency is best, based on the calculated cost function values, as theoptimal inter prediction mode. The motion prediction/compensation unit32 outputs the prediction image data generated in the optimal interprediction mode and the cost function value of the optimal interprediction mode, to the prediction image/optimal mode selecting unit 33.Further, the motion prediction/compensation unit 32 outputs predictionmode information relating to the optimal inter prediction mode to thelossless encoding unit 16 and information generating unit 19.

The prediction image/optimal mode selecting unit 33 compares the costfunction value supplied from the intra prediction unit 31 to the costfunction value supplied from the motion prediction/compensation unit 32,and selects the one of which the cost function value is less than theother as the optimal mode where the encoding efficiency will be best.Also, the prediction image/optimal mode selecting unit 33 outputs theprediction image data generated in the optimal mode to the subtractingunit 13 and adding unit 23. Further, the prediction image/optimal modeselecting unit 33 outputs information indicating whether the optimalmode is the intra prediction mode or the inter prediction mode to thelossless encoding unit 16 and information generating unit 19. Note thatthe prediction image/optimal mode selecting unit 33 performs switchingof the intra prediction or inter prediction in increments of slices.

<2. Operation of Image Encoding Device>

With the image encoding device, encoding processing is performed withthe macroblock size extended beyond that with the H.264/AVC format, forexample. FIG. 5 exemplarily illustrates the hierarchical structure ofcoding units. Note that FIG. 5 illustrates a case where the maximum sizeis 128 pixels×128 pixels, and the hierarchical depth (Depth) is “5”. Forexample, in the event that the hierarchical depth is “0”, a 2N×2N (N=64pixels) block is coding unit CU0. Also, when split flag=1, the codingunit CU0 is divided into four independent N×N blocks, with the N×Nblocks being blocks of one hierarchical level lower. That is to say, thehierarchical depth is “1”, and 2N×2N (N=32 pixels) blocks are codingunit CU1. In the same way, when split flag=1, this is divided into fourindependent blocks. Further, when the depth “4” which is the deepesthierarchical level, 2N×2N (N=4 pixels) blocks are coding unit CU4, and 8pixels×8 pixels are the smallest size for coding units CU. Also, withHEVC, prediction unit (PU: Prediction Unit) which is a basic unit fordividing coding units and predicting, and transform unit (TU: TransformUnit) which is a basic unit for transformation and quantization, aredefined.

Next, operations of the image encoding device will be described withreference to the flowchart in FIG. 6. In step ST11, the A/D convertingunit 11 performs A/D conversion on the input image signals.

In step ST12, the screen rearranging buffer 12 performs imagerearranging. The screen rearranging buffer 12 stores image data suppliedfrom the A/D converting unit 11 and performs rearranging from an orderfor displaying the pictures to an order for encoding.

In step ST13, the subtracting unit 13 generates prediction error data.The subtracting unit 13 calculates the difference between the image dataof images rearranged in step ST12 and prediction image data selected atthe prediction image/optimal mode selecting unit 33 to generateprediction error data. The data amount of the prediction error data issmaller than that of the original image data. Therefore, data amount canbe compressed in comparison with a case where the image is encoded as itis.

In step ST14, the orthogonal transform unit 14 performs orthogonaltransform processing. The orthogonal transform unit 14 performsorthogonal transform on the prediction error data supplied from thesubtracting unit 13. Specifically, orthogonal transform such as discretecosine transform, Karhunen-Loéve transform, and the like are performedas to the prediction error data to output transform coefficient data.

In step ST15, the quantization unit 15 performs quantization processing.The quantization unit 15 quantizes transform coefficient data. Ratecontrol is performed at the time of quantization, as illustrated in thelater-described processing in step ST25.

The inverse quantization unit 21 performs inverse quantizationprocessing in step ST16. The inverse quantization unit 21 performsinverse quantization on the transform coefficient data quantized by thequantization unit 15 with properties corresponding to the properties ofthe quantization unit 15.

In step ST17, the inverse orthogonal transform unit 22 performs inverseorthogonal transform processing. The inverse orthogonal transform unit22 performs inverse orthogonal transform on the transform coefficientdata subjected to inverse quantization by the inverse quantization unit21 with properties corresponding to the properties of the orthogonaltransform unit 14.

In step ST18, the adding unit 23 generates reference image data. Theadding unit 23 adds the prediction image data supplied from theprediction image/optimal mode selecting unit 33 and the data afterinverse orthogonal transform of the corresponding position to thisprediction image, to generate decoded data (reference image data).

In step ST19, the deblocking filter 24 performs filtering processing.The deblocking filter 24 filters decoded image data output from theadding unit 23 and removes block distortion.

In step ST20, the frame memory 26 stores the reference image data. Theframe memory 26 stores the decoded image data after filtering processing(reference image data).

In step ST21, the intra prediction unit 31 and motionprediction/compensation unit 32 each performs prediction processing.That is to say, the intra prediction unit 31 performs intra predictionprocessing of the intra prediction mode, and the motionprediction/compensation unit 32 performs motion prediction/compensationprocessing of the inter prediction mode. The prediction processing isdescribed below with reference to FIG. 7, in which the predictionprocessing with all candidate prediction modes is each performed, andcost function values with all the candidate prediction mode are eachcalculated by this processing. Further, based on the calculated costfunction values, the optimal intra prediction mode and optimal interprediction mode are selected, and the prediction image and the costfunction and prediction mode information generated in the selectedprediction mode are supplied to prediction image/optimal mode selectingunit 33.

In step ST22, the prediction image/optimal mode selecting unit 33selects the prediction image data. The prediction image/optimal modeselecting unit 33 decides in the optimal mode of which the encodingefficiency is best, based on each cost function value output from theintra prediction unit 31 and motion prediction/compensation unit 32.That is to say, the prediction image/optimal mode selecting unit 33decides the coding unit where the encoding efficiency is best from eachof the hierarchical levels illustrated in FIG. 5 for example, the blocksize of prediction untie in this coding unit, and which of intraprediction and inter prediction to perform. Further, the predictionimage/optimal mode selecting unit 33 outputs the prediction image dataof the decided optimal mode to the subtracting unit 13 and adding unit23. This prediction image data is used for the computation of step ST13and ST8, as described above.

In step ST23, the lossless encoding unit 16 performs lossless encodingprocessing. The lossless encoding unit 16 performs lossless encoding onthe quantization data output from the quantization unit 15. That is,lossless encoding such as variable length encoding or arithmeticencoding is performed as to the quantization data to be made datacompression. Also, the lossless encoding unit 16 performs losslessencoding of prediction mode information and the like corresponding tothe prediction image data selected in step ST22, and lossless encodeddata such as prediction mode information and the like is included insteam information generated by performing lossless encoding ofquantization data.

In step ST24, the storage buffer 17 performs storage processing. Thestorage buffer 17 stores stream information output from the losslessencoding unit 16. The stream information stored in this storage buffer17 is read out appropriately and is transmitted to the decoding sideover the transmission path.

In step ST25, the rate control unit 18 performs rate control. The ratecontrol unit 18 controls, in the case of storing stream information inthe storage buffer 17, the rate of the quantization operation of thequantization unit 15 so that overflow or underflow does not occur in thestorage buffer 17.

Next, prediction processing in step ST21 in FIG. 6 will be describedwith reference to the flowchart in FIG. 7.

In step ST31, the intra prediction unit 31 performs intra predictionprocessing. The intra prediction unit 31 performs intra prediction onthe image of the prediction unit to be encoded, in all candidate intraprediction modes. Note that, for the image data of the decoded imagereferred to in intra prediction, the decoded image data before beingsubjected to deblocking filter processing by the deblocking filter 24 isused. Due to this intra prediction processing, intra prediction isperformed in all candidate intra prediction modes, and a cost functionvalue is calculated for all candidate intra prediction modes. One intraprediction mode of which the encoding efficiency is best is thenselected from all intra prediction modes, based on the calculated costfunction values.

In step ST32, the motion prediction/compensation unit 32 performs interprediction processing. The motion prediction/compensation unit 32performs inter prediction processing of all candidate inter-predictionmodes using decoded image data after deblocking filter processing storedin the frame memory 26. Due to this inter prediction processing,prediction processing is performed in all candidate inter predictionmodes, and cost function values are calculated for candidate interprediction modes. One inter prediction mode of which encoding efficiencyis the best is then selected from all inter prediction modes, based onthe calculated cost function values.

The intra prediction processing in step ST31 in FIG. 7 will be describedwith reference to the flowchart in FIG. 8.

In step ST41, the intra prediction unit 31 performs intra prediction ofeach prediction mode. The intra prediction unit 31 generates predictionimage data in each intra prediction mode, using decoded image databefore and after blocking filter processing.

In step ST42, the intra prediction unit 31 calculates cost functionvalues in each prediction mode. Calculation of cost function values isperformed, as stipulated in JM (Joint Model) which is the referencesoftware in the H.264/AVC format, based on either technique of the HighComplexity mode or Low Complexity mode.

That is to say, in the High Complexity mode, up to the lossless encodingprocessing is tentatively performed as to all candidate predictionmodes, and the cost function value represented by the followingExpression (1) are calculated as to each prediction mode.

Cost(Mode∈Ω)=D+λ·R  (1)

Ω represents an whole set of candidate prediction modes to encode theimage of this prediction unit. D represents difference energy(distortion) between a decoded image and an input image in the eventthat encoding has been performed in prediction mode. R is generated codeamount including orthogonal transform coefficient, prediction modeinformation, and so forth, and λ is a Lagrange multiplier given as afunction of quantization parameter QP. That is, in High Complexity Mode,up to the lossless encoding processing is tentatively performed for allcandidate prediction modes as the processing of step ST42, and the costfunction values represented by Expression (1) above are calculated foreach prediction mode.

On the other hand, in the Low Complexity mode, generating of predictionimages and generating of header bits including difference motion vectorsand prediction mode information and so forth, is performed for allcandidate prediction modes, and cost function values represented by thefollowing Expression (2) are calculated.

Cost(Mode∈Ω)=D+QP2Quant(QP)·Header Bit  (2)

Q represents the whole set of candidate prediction modes to encode theimage of this prediction unit. D represents difference energy(distortion) between a decoded image and an input image in the eventthat encoding has been performed in prediction mode. Header_Bit is aheader bit for the prediction mode, and QP2Quant is a function which isgiven as a function of quantization parameter QP. That is, in the LowComplexity Mode, the cost function value represented by Expression (2)above is calculated for each prediction mode, using generating ofprediction image and header bits such as motion vectors and predictionmode information and so forth, as the processing of step ST42.

In step ST43, the intra prediction unit 31 decides the optimal intraprediction mode. The intra prediction unit 31 selects an intraprediction mode of which the cost function value is the smallest basedon the cost function values calculated in step ST42, which is decided tothe optimal intra prediction mode.

Next, the inter prediction processing of step ST32 in FIG. 7 will bedescribed with reference to the flowchart in FIG. 9.

In step ST51, the motion prediction/compensation unit 32 performs motiondetection processing. The motion prediction/compensation unit 32 detectsmotion vectors and advances to step ST52.

In step ST52, the motion prediction/compensation unit 32 performs motioncompensation processing. The motion prediction/compensation unit 32performs motion compensation using reference image data based on themotion vectors detected in step ST51, and generates prediction imagedata.

In step ST53, the motion prediction/compensation unit 32 performscalculation of cost function values. The motion prediction/compensationunit 32 calculates cost function values as described above, using theinput image data of the prediction image which is to be encoded, and theprediction image data generated in step ST52 and so forth, and advancesto step ST54.

The motion prediction/compensation unit 32 decides an optimal interprediction mode. The motion prediction/compensation unit 32 performs theprocessing from step ST51 through ST53 for each inter prediction mode.The motion prediction/compensation unit 32 distinguishes the referenceindex where the cost function value calculated for each prediction modeis the smallest value, the block size of the coding unit, and the blocksize of the prediction unit in this coding unit, and decodes the optimalinter prediction mode. Note that with decision of the mode where thecost function is the smallest, the cost function value in a case ofhaving performed inter prediction in skip mode is also used.

Also, in the event that the optimal inter prediction mode has beenselected as the optimal prediction mode at the prediction image/optimalmode selecting unit 33, the motion prediction/compensation unit 32generates prediction image data such that prediction image data of theoptimal inter prediction mode can be supplied to the subtracting unit 13and the adding unit 23.

<3. Generating Operation of Identification Information and DifferenceInformation Based on Quantization Parameters>

In the above-described image encoding processing, the image encodingdevice 10 sets quantization parameters that suitable quantization isperformed for each block according to the complexity of the image. Also,the image encoding device 10 generates identification information anddifference information and includes these in the stream information, toimprove encoding efficiency of the quantization parameters used in thequantization processing in step ST15.

Next, description will be made regarding generating of theidentification information and difference information. The rate controlunit 18 sets the quantization parameters using the code amount controlformat stipulated with TM5 in MPEG2, for example.

With the code amount control format stipulated with TM5 in MPEG2, theprocessing of step 1 through step 3 is illustrated.

In step 1, the amount of code to be allocated to each picture within aGOP (Group of Pictures) is distributed to the pictures not encoded yet,including pictures for allocation, based on an allocation bit amount R.This distribution is repeated in the order of encoded pictures withinthe GOP. At this time, code amount allocation to each picture isperformed using the following two assumptions.

The first assumption is that the product of mean quantized scale codeand generated code amount, used at the time of encoding each picture,will be constant for each picture type, unless the screen changes.

Accordingly, after encoding each picture, parameters X_(I), X_(P), andX_(B) (Global Complexity Measure) representing the complexity of thescreen are updated by Expressions (3) through (5). The relation betweenthe quantization scale code and generated code amount can be estimatedby these parameters.

X _(I) =S _(T) ·Q _(I)  (3)

X _(P) =S _(P) ·Q _(P)  (4)

X _(B) =S _(B) ·Q _(B)  (5)

Here, S_(I), S_(P), and S_(B), are generated code bits at the time ofpicture encoding, and Q_(I), Q_(E), and Q_(B), are mean quantizationscale code at the time of picture encoding. Also, initial values arevalues illustrated by the Expressions following (6), (7), and (8), usingbit_rate [bits/sec] which is the target code amount.

X _(I)=160×bit_rate/15  (6)

X _(P)=160×bit_rate/15  (7)

X _(B)=160×bit_rate/15  (8)

The second assumption is that the overall image quality will beconstantly optimized when the ratios K_(P) and K_(B) for quantizationscale code of the P and B pictures, with the quantization scale code ofthe I picture as a reference, are such as stipulated in Expression (9).

K _(P)=1.0; K _(B)=1.4  (9)

That is to say, the quantization scale code for B pictures is constantlyset to 1.4 times the quantization scale code for I and P pictures. Thisassumes that by making the B pictures to be quantized somewhat coarserthan as compared with the I and P pictures and thereby adding the codeamount conserved with the B pictures to the I and P pictures, the imagequality of the I and P pictures will be improved, and also the imagequality of the B pictures referring to these will be improved.

According to the above two assumptions, the allocated code amounts(T_(I), T_(P), T_(B)) as to each picture in the GOP are the valuesindicated in the Expressions (10), (11), and (12). Note thatpicture_rare indicates the number of pictures displayed per second inthis sequence.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 1} \right\rbrack & \; \\{T_{I} = {\max \left\{ {\frac{R}{1 + \frac{N_{P}X_{P}}{X_{I}K_{P}} + \frac{N_{B}K_{B}}{X_{I}K_{B}}},\frac{bit\_ rate}{8 \times {picture\_ rate}}} \right\}}} & (10) \\{T_{P} = {\max \left\{ {\frac{R}{N_{P} + \frac{N_{B}K_{P}X_{B}}{K_{B}X_{P}}},\frac{bit\_ rate}{8 \times {picture\_ rate}}} \right\}}} & (11) \\{T_{I} = {\max \left\{ {\frac{R}{N_{B} + \frac{N_{P}K_{B}X_{P}}{K_{P}X_{B}}},\frac{bit\_ rate}{8 \times {picture\_ rate}}} \right\}}} & (12)\end{matrix}$

Now, N_(P), N_(B) are the number of P and B pictures not encoded withinthe GOP. That is to say, of the pictures not encoded within the GOP,with regard to pictures to which allocation is to be performed and thoseof different picture types, estimation is made regarding how many timesthe generated code amount of the picture for allocation the code amountgenerated by those pictures will be, under the above-described imagequality optimization conditions. Next, how many pictures to be encodedworth of code amount the estimated generated code amount which theentirety of unencoded pictures generates is equivalent to is obtained.For example, N_(P)X_(P)/X_(I)K_(P), which is the second term of thedenominator of the first argument in the expression relating to Ti,expresses how many I pictures that the N_(P) unencoded pictures withinthe GOP are worth. Also, this is obtained by multiplying N_(P) by afraction S_(P)/S_(I) of the generated code amount of the I picture as tothe generated code amount of the P pictures, and expressing by X_(I),X_(P), and X_(B) as described above.

The bit amount as to the pictures for allocation is obtained by dividingthe allocation code amount R as to unencoded pictures by the number ofpictures. Note that a lower limit is set to that value, however, takinginto consideration the overhead code amount for the header and so forth.

Based on the allocated code amount thus obtained, the code amount R tobe allocated to unencoded pictures within the GOP is updated byexpression (13) each time each picture is encoded following steps 1 and2.

R=R−S _(I,P,B)  (13)

Also, at the time of encoding the first picture of the GOP, R is updatedby the following Expression (14).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 2} \right\rbrack & \; \\{R = {\frac{{bit\_ rate} \times N}{picture\_ rate} - R}} & (14)\end{matrix}$

where N is the number of pictures within the GOP. Also, the initialvalue of R at the beginning of the sequence is 0.

Next, description will be made regarding step 2. In step 2, aquantization scale code for actually matching the allocation codeamounts (T_(I), T_(P), T_(B)) as to each picture, obtained in step 1, tothe actual code amount, is obtained. The quantization scale code isobtained by feedback control in macroblock increments for each picturetype, based on the capacity of three types of virtual buffersindependently established.

First, before encoding of a j'th macroblock, the occupation amounts ofthe virtual buffers are obtained by Expressions (15) through (17).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 3} \right\rbrack & \; \\{d_{j}^{I} = {d_{0}^{I} + B_{j - 1} - \frac{T_{I} \times \left( {j - 1} \right)}{MBent}}} & (15) \\{d_{j}^{P} = {d_{0}^{P} + B_{j - 1} - \frac{T_{P} \times \left( {j - 1} \right)}{MBent}}} & (16) \\{d_{j}^{B} = {d_{0}^{B} + B_{j - 1} - \frac{T_{B} \times \left( {j - 1} \right)}{MBent}}} & (17)\end{matrix}$

d₀ ^(I), d₀ ^(P), and doe are the initial occupation amounts of thevirtual buffers, B_(j) is the generated bit amount from the head of thepicture to the j'th macroblock, and MB_(cnt) is the number ofmacroblocks within a single picture.

The occupation amounts of the virtual buffers at the time of endingencoding of each picture (dMB_(cnt) ^(I), dMB_(cnt) ^(P), dMB_(cnt)^(B)) are used as the initial values of the virtual buffers occupationamount for the next picture (d₀ ^(I), d₀ ^(P), d₀ ^(B)) for the samepicture type, respectively.

Next, a reference quantization scale code Q_(j) for the j'th macroblockis calculated by Expression (18).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 4} \right\rbrack & \; \\{Q_{j} = \frac{d_{j} \times 31}{r}} & (18)\end{matrix}$

r is a parameter controlling response speed of a feedback group, calleda reaction parameter, and is obtained by Expression (19).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 5} \right\rbrack & \; \\{r = {2 \times \frac{bit\_ rate}{picture\_ rate}}} & (19)\end{matrix}$

Note that the initial value of the virtual buffer at the beginning ofthe sequence is obtained by Expression (20).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 6} \right\rbrack & \; \\{{d_{0}^{I} = {10 \times \frac{r}{31}}},{d_{0}^{P} = {K_{p}d_{0}^{I}}},{d_{0}^{B} = {K_{B}d_{0}^{I}}}} & (20)\end{matrix}$

Next, step 3 will be described. Activity is obtained from Expressions(21) through (23) using luminance signal pixel values of the originalimage, e.g., using a pixel values of a total of eight blocks of four 8×8blocks in frame DCT mode and four 8×8 blocks in field DCT encoding mode.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 7} \right\rbrack & \; \\{{act}_{j} = {1 + {\min\limits_{{{sblk} = 1},8}\left( {{var}\mspace{14mu} {sblk}} \right)}}} & (21) \\{{{var}\mspace{14mu} {sblk}} = {\frac{1}{64}{\sum\limits_{K = 1}^{64}\left( {P_{k} - \overset{\_}{P}} \right)^{2}}}} & (22) \\{\overset{\_}{P} = {\frac{1}{64}{\sum\limits_{K = 1}^{64}P_{k}}}} & (23)\end{matrix}$

The var_sblk in Expression (21) is the sum of squares of differencebetween the image data of each pixel and the average value thereof, sothe more complex the images of these 8×8 blocks are, the greater thevalue is. P_(k) in Expressions (22) and (23) is in-block pixel values ofluminance signals of the original image. The reason that the minimalvalue (min) is assumed in Expression (22) is to make quantization finerin the event that there is even a partially smooth portion within the16×16 macroblock. Further, a normalized activity N_(actj) where thevalue thereof is within the range of 0.5 to 2 is obtained by Expression(24).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 8} \right\rbrack & \; \\{{Nact}_{j} = \frac{{2 \times {act}_{j}} + {avg\_ act}}{{act}_{j} + {2\; \times {avg\_ act}}}} & (24)\end{matrix}$

avg_act is the average value of activity up to the picture encodedimmediately before. A quantization scale code mquant_(j) which takesinto consideration visual properties is obtained by Expression (25)based on reference quantization scale code Q_(j).

[Math. 9]

mquant_(j) =Q _(j) ×Nact_(j)  (25)

The rate control unit 18 outputs the quantization scale code mquant_(j)calculated as described above as a quantization parameter. Also, aquantization parameter is generated for the macroblock situated at theslice boundary in the same way as with macroblocks situated at otherthan the slice boundary, with the same technique. Note that quantizationparameters are not restricted to cases of being decided based onactivity as described above, and may be decided such that the costfunction value is smaller.

Note that with the description of the rate control method stipulatedwith TM5 in MPEG2 described above, a case where processing is performedin increments of macroblocks is described. Accordingly, by performingsimilar processing in increments of blocks regarding which quantizationparameters can be switched, quantization parameters can be set for eachof the blocks regarding which quantization parameters can be switched.

Next, description will be made regarding generating operations ofinformation used to improve encoding efficiency of quantizationparameters. The information generating unit 19 takes encodedquantization parameters spatially or temporally adjacent to the block tobe encoded as selection candidates. The information generating unit 19also selects a quantization parameter from selection candidates inaccordance with a quantization parameter set as to the block to beencoded, and takes this as a prediction quantization parameter. Theinformation generating unit 19 further generates identificationinformation for selecting a prediction quantization parameter fromselection candidates, and difference information indicating thedifference between the prediction quantization parameter and aquantization parameter set to the block to be encoded.

FIG. 10 is a diagram for describing operations of the informationgenerating unit, illustrating a frame to be encoded, and an encodedframe which is temporally closest in display order. We will say that thequantization parameter of the block to be encoded in the frame to beencoded is, for example, “QP_0”. Also, we will say that the quantizationparameters of the block adjacent to the left is, for example, “QP_A”. Inthe same way, we will say that the quantization parameter of the blocksadjacent above, to the upper right, to the upper left, and to the lowerleft are, for example, “QP_B”, “QP_C”, “QP_D”, and “QP_E”. Also, we willsay that the quantization parameters of the block temporally adjacent is“QP_T”. Note that when encoding the block to be encoded in the frame tobe encoded, the quantization parameters “QP_A” through “QP_E” and “QP_T”are stored in the quantization parameter memory unit 191. Also, we willsay that each block is the smallest increment block regarding whichquantization parameters can be changed.

The difference computing unit 192 takes quantization parameters ofencoded block adjacent to the block to be encoded as selectioncandidates, selects from the selection candidates the quantizationparameter of which the difference as to the quantization parameter setto the block to be encoded is the smallest, and takes this as aprediction quantization parameter. The difference computing unit 192generates identification information for selecting the predictionparameter from the selection candidates, and difference informationindicating the difference between the prediction quantization parameterand the quantization parameter of the block to be encoded.

FIG. 11 is a diagram illustrating an operation example of theinformation generating unit. Note that cases where no quantizationparameter is set to a block due to being a skip block or with noresidual information are indicated by a “-”.

With the block to be encoded as block BK0, the quantization parametersof the encoded blocks are “QP_A=32”, “QP_B=40”, “QP_C=40”, “QP_D=35”,“QP_E=-”, and “QP_T=31”. Here, the information generating unit 19excludes blocks where no quantization parameter is set to a block due tobeing a skip block or a block with no residual information, and blockswhere quantization parameters are redundant, from candidates.Accordingly, the selection candidates are the encoded blocks of thequantization parameters “QP_A=32”, “QP_B=40”, “QP_D=35”, and “QP_T=31”.Also, the information generating unit 19 sets identificationinformation, index Nos. for example, to the selection candidatesbeforehand. The identification information may be set to adjacentencoded blocks alone, or may be set to the quantization parameters ofthe adjacent encoded blocks.

In the event of setting identification information to adjacent encodedblocks, the information generating unit 19 sets index numbers in orderof array with the adjacent encoded blocks in a predetermined order ofarray. The predetermined order of array is, for example, an order ofarray where one of an encoded block adjacent to the left side, anencoded block adjacent above, and an encoded block temporally adjacent,is given priority. Also, the information generating unit 19 may becapable of switching the order of array. In the event of being capableof switching the order of array, information indicating what sort oforder of array is included in the stream information. Also, the losslessencoding unit 16 and information generating unit 19 perform settings andlossless encoding of the identification information such that there isless code amount when encoding the identification information of theblock given priority.

The difference computing unit 192 selects a candidate from the selectioncandidates where the difference as to the quantization parameter of theblock to be encoded is smallest, and uses the identification informationset to the selected candidate, thereby generating identificationinformation for selecting a prediction quantization parameter from theselection candidates. Also, the difference computing unit 192 generatesdifference information indicating the difference between the predictionquantization parameters which is the selected candidate quantizationparameter, and the quantization parameter of the block to be encoded.For example, in the event of giving priority to the encoded blockadjacent to the left side in FIG. 11, the information generating unit 19sets “0 (index No.): block of QP_A”, “1: block of QP_B”, “2: block ofQP_B”, and “3: block of QP_T”. Also, if we way that the quantizationparameter of the block to be encoded is “33” for example, the differencecomputing unit 192 sets the index No. of the block where the differenceas to the quantization parameter of the block to be encoded is thesmallest, to identification information “0 (index No.)”. Also, thedifference computing unit 192 generates difference information “1(=33−32)” indicating the difference between the prediction quantizationparameter and the quantization parameter of the block to be encoded.

By setting identification information to blocks to be encoded in thisway, the encoding efficiency of quantization parameters can be improved.For example, if the block at the left side is given priority and theblock order is quantization parameters “QP_A”, “QP_B”, “QP_C”, “QP_D”,“QP_E”, “QP_T”, data amount will be small with images where there aremore blocks to be encoded that are similar to the image of the block tothe left side. Also, if the block above is given priority and the blockorder is quantization parameters “QP_B”, “QP_A”, “QP_C”, “QP_D”, “QP_E”,“QP_T”, data amount will be small with images where there are moreblocks to be encoded that are similar to the image of the block above.Further, if the block temporally adjacent is given priority and theblock order is quantization parameters “QP_T”, “QP_A”, “QP_B”, “QP_C”,“QP_D”, “QP_E”, data amount will be small with images where there aremore blocks to be encoded that are similar to the image temporallyadjacent, i.e., more still subjects.

In a case of setting identification information as to quantizationparameters of adjacent encoded blocks, the information generating unit19 sets index Nos. with the adjacent encoded blocks in a predeterminedorder of array. For example, the information generating unit 19 setsindex Nos. in order of quantization parameters with small parametervalues. That is to say, in the case of FIG. 11, the informationgenerating unit 19 sets index Nos. such as “0 (index No.): 32(quantization parameter)”, “1:40”, “2:35”, “3:31”.

The difference computing unit 192 selects a candidate from the selectioncandidates where the difference as to the quantization parameter of theblock to be encoded is smallest, and uses the identification informationset to the selected candidate, thereby generating identificationinformation for selecting a prediction quantization parameter from theselection candidates. Also, the difference computing unit 192 generatesdifference information indicating the difference between the predictionquantization parameter and the quantization parameter of the block to beencoded. For example, if we say that the quantization parameter of theblock to be encoded is “33”, the difference computing unit 192 generatesdifference information “1 (=33-32)” as identification information.

Also, in the event that there are not selection candidates, thedifference computing unit 192 generates difference informationindicating the difference between the quantization parameter SliceQPY ofthe initial value in the slice and the set quantization parameter.

FIG. 12 is a flowchart illustrating processing regarding quantizationparameters in encoding. In step ST61, the image encoding device 10generates information for obtaining a quantization parameters unitminimum size (MinQpUnitSize). The quantization parameters unit minimumsize is the smallest size where quantization parameters can beadaptively switched.

The image encoding g device 10 uses, as information for obtaining thequantization parameters unit minimum size (MinQpUnitSize), difference asto a transform unit minimum size (MinTransformUnitSize), for example.

The quantization parameters unit minimum size (MinQpUnitSize) isdetermined by Expression (26).

MinQpUnitSize=1<<(log 2_min_transform_unit_size_minus2+log2_min_qp_unit_size_offset+2)  (26)

Note that “log 2_min_transform_unit_size_minus2” is a parameter fordeciding the transform unit minimum size (MinTransformUnitSize).

The transform unit minimum size (MinTransformUnitSize) is decided byExpression (27).

MinTransformUnitSize=1<<(log 2_min_transform_unit_size_minus2+2)  (27)

The difference between the quantization parameter unit minimum size(MinQpUnitSize) and the transform unit minimum size(MinTransformUnitSize) is, as can be clearly understood from Expressions(26) and (27), is equivalent to “log 2_min_qp_unit_size_offset”. Notethat quantization parameters are used in increments of transform units(TU). That is to say, a quantization parameter is unchanged within atransform unit.

Also, the quantization parameter unit minimum size (MinQpUnitSize) maybe decided in accordance with coding unit size. In this case, the imageencoding unit 10 uses, for example, information stipulating the minimumsize of the coding unit CU (log 2_min-coding_block_size_minus3), andmaximum size of the coding unit CU (log 2_diffmax_min_coding_block_size). Note that the maximum size of the codingunit CU “log 2MaxCUSize” is as illustrated in Expression (28).

log 2MaxCUSize=log 2_min_coding_block_size_minus 3+3+log2_diff_max_min_coding_block_size  (28)

The logarithmic value of the quantization parameter unit minimum size(log 2MinQpUnitSize) is decided by Expression (29).

log 2MinQpUnitSize=log 2_min_coding_block_size_minus 3+3+log2_diff_max_min_coding_block_size−log 2_min_qp_unit_size_offset  (29)

Accordingly, setting “log 2_min_qp_unit_size_offset” so as to be greatermakes the quantization parameter unit minimum size smaller. For example,in a case where the smallest size of a coding unit CU is “8×8” and thegreatest size is “64×64”, setting “log 2_min_qp_unit_size_offset” to “1”makes the quantization parameter unit minimum size to be “32×32”. Also,setting “log 2_min_qp_unit_size_offset” to “2” makes the quantizationparameter unit minimum size to be “16×16”.

In step ST62, the image encoding device 10 performs processing ofincluding the generated information in the stream information. The imageencoding device 10 includes “log 2_min_qp_unit_size_offset”, and “log2_min_qp_unit_size_offset” which is a parameter to decide the transformunit minimum size (MinTransformUnitSize), in the stream information, andadvances to step ST63. Also, in the event of deciding the quantizationparameter unit minimum size in accordance with the coding unit size,“log 2_min_coding_block_size_minus3”, “log2_diff_max_min_coding_block_size”, and “log 2_min_qp_unit_size_offset”are included in the stream information. The image encoding device 10includes the generated information in a sequence parameter set (SPS:sequence parameter set) defined as a syntax of RBSP (raw byte sequencepayload), for example. Note that FIG. 13 exemplarily illustrates asequence parameter set.

In step ST63, the image encoding device 10 determines whether or not aframe to encode exists. In the event that a frame to encode exists, theimage encoding device 10 advances to step ST 64 and performs frameencoding processing illustrated in FIG. 14, and if this does not exist,ends encoding processing.

In the frame encoding processing in FIG. 14, in step ST71 the imageencoding device 10 determines whether or not a slice to encode exists.In the event that a slice to encode exists, the image encoding device 10advances to step ST72, and if this does not exist, ends the encodingprocessing of the frame.

In step ST72, the image encoding device 10 decides the quantizationparameters of the slice to encode. The image encoding device 10 decidesthe quantization parameter of the initial value in the size so as to bea target code amount, and advances to step ST73.

In step ST73, the image encoding device 10 calculates “slice_qp_delta”.The quantization parameter SliceQPY of the initial value in the slicehas the relation illustrated in Expression (30), with“pic_init_qp_minus26” being set by the user or the like beforehand.Accordingly, the image encoding device 10 calculates “slice_qp_delta” soas to be the quantization parameter decided in step ST72, and advancesto step ST74.

SliceQPY=26+pic_init_qp_minus26+slice_qp_delta   (30)

In step ST74 the image encoding device 10 includes “slice_qp_delta” and“picinit_qp_minus26” in the stream information. The image encodingdevice 10 includes the calculated “slice_qp_delta” in the header slice,for example, of the stream information. Also, the image encoding device10 includes the “pic_initqp_minus26” that has been set, in the pictureparameter set, for example, of the stream information. By thus includingthe “slice_qp_delta” and “pic_initqp_minus26” in the stream information,the image decoding device which performs decoding of the streaminformation can calculate the quantization parameter SliceQPY of theinitial value in the slice by performing the computation of Expression(30). Note that FIG. 15 exemplarily illustrates a sequence parameterset, and FIG. 16 a slice header.

In step ST75, the image encoding device 10 performs slice encodingprocessing. FIG. 17 is a flowchart illustrating performs slice encodingprocessing.

In step ST8 of FIG. 17, the image encoding device 10 determines whetheror not a coding unit CU to encode exists. In the event that a codingunit regarding which encoding processing has not been performed yetexists in the slice to be encoded, the image encoding device 10 advancesto step ST82. Also, in the event that encoding processing of all codingunits in the slice has been completed, the image encoding device 10 endsthe slice encoding processing.

In step ST82, the image encoding device 10 determines whether or not atransform unit TU exists in the coding unit CU to be encoded. In theevent that a transform unit exists, the image encoding device 10advances to step ST83, and in the event that a transform unit does notexist, advances to step ST87. For example, in the event that allcoefficients to be quantized using a quantization parameter are “0”, orin a case of a skip block, the flow advances to step ST87.

In step ST83, the image encoding device 10 decides the quantizationparameter of the coding unit CU to be encoded. The rate control unit 18of the image encoding device 10 decides the quantization parameter inaccordance with the complexity of the image of the coding unit asdescribed above, or such that the cost function value is small, andadvances to step ST84.

In step ST84, the image encoding device 10 sets identificationinformation to the selection candidates. The information generating unit19 of the image encoding device 10 takes quantization parameters ofencoded codings spatially or temporally peripheral to the coding unit tobe encoded, as selection candidates. Also, in the event that noquantization parameters are set to the block due to being a skip blockor having no residual information, or in the event that a quantizationparameter is equal to another candidate, the information generating unit19 excludes these from selection candidates. The image encoding device10 sets identification information, e.g., index (ref_qp_block_index) tothe selection candidates, and advances to step ST85.

In step ST85, the image encoding device 10 generates identificationinformation and difference information. The information generating unit19 of the image encoding device 10 selects from the selection candidatesa candidate where the difference as to the quantization parameter of thecoding unit to be encoded is smallest, and takes this as a predictionquantization parameter. The information generating unit 19 generatesidentification information by using the index (ref_qp_block_index) ofthe selected candidate as identification information for selecting theprediction quantization parameter from the selection candidates. Also,the information generating unit 19 takes the difference (qb_qp_delta)between the prediction quantization parameter and the quantizationparameter of the coding unit to be encoded, as difference information,and advances to step ST86. Now, with the prediction quantizationparameter indicated by the index (ref_qp_block_index) of the determinedcandidate as “ref_qp(ref_qp_block_index)”, the quantization parameter ofthe coding unit to be encoded (CurrentQP) exhibits the relationshipindicated in Expression (31).

CurrentQP=qb_qp_delta+ref_qp(ref_qp_block_index)   (31)

In step ST86, the image encoding device 10 includes the identificationinformation and difference information in the stream information. Thelossless encoding unit 16 of the image encoding device 10 performslossless encoding of the identification information and differenceinformation generated at the information generating unit 19, includes inthe stream information, and advances to step ST87.

In step ST87 the image encoding device 10 uses the decided quantizationparameter to perform quantization of the coding unit with thequantization unit 15, and returns to step ST81.

Thus, the image encoding device 10 selects, from quantization parametersof encoded blocks spatially or temporally adjacent to a block to beencoded, a candidate where the difference as to the quantizationparameter of the block to be encoded is the smallest, as a predictionquantization parameter. Also, the image encoding device 10 generatesidentification information corresponding to the selected quantizationparameter. Further, the image encoding device 10 generates differenceinformation indicating difference between the prediction quantizationparameter and the quantization parameter of the block to be encoded. Theimage encoding device 10 includes the generated identificationinformation and difference identification in stream information. Thus,since a candidate where the difference is the smallest is selected asthe prediction quantization parameter, the difference between theprediction quantization parameter and the quantization parameter of theblock to be encoded can be prevented from becoming a great value.Accordingly, the image encoding device 10 can improve the encodingefficiency of quantization parameters.

<4. Configuration of Image Decoding Device>

Next, an image decoding device which performs decoding processing ofstream information output from the image encoding device will bedescribed. The encoded stream generated by encoding an input image issupplied to the image decoding device via a predetermined transmissionpath, recording medium, or the like, and decoded.

FIG. 18 shows a configuration for an image decoding device whichperforms decoding processing of stream information. The image decodingdevice 50 includes a storage buffer 51, a lossless decoding unit 52, aninverse quantization unit 53, an inverse orthogonal transform unit 54,an adding unit 55, a deblocking filter 56, a screen rearranging buffer57, and a digital/analog converting unit (D/A converting unit) 58.Furthermore, the image decoding device 50 includes a quantizationparameter calculating unit 59, frame memory 61, an intra prediction unit71, a motion compensation unit 72, an a selector 73.

The storage buffer 51 stores the stream information which has beentransmitted. The lossless decoding unit 52 decodes the streaminformation supplied from the storage buffer 51 by a formatcorresponding to the encoding format of the lossless encoding unit 16 inFIG. 3.

The lossless decoding unit 52 operates as an information obtaining unitand obtains various types of information from the stream information.For example, the lossless decoding unit 52 outputs prediction modeinformation obtained by decoding the stream information to the intraprediction unit 71 and motion compensation unit 72. Also, the losslessdecoding unit 52 outputs difference motion vectors, threshold values, orthreshold value generating information, obtained by decoding the streaminformation, to the motion compensation unit 72. Also, the losslessdecoding unit 52 outputs information related to quantization parametersobtained by decoding the stream information, e.g., differenceinformation and the like, to the quantization parameter calculating unit59. Further, the lossless decoding unit 52 outputs the quantization dataobtained by decoding the stream information to the inverse quantizationunit 53.

The inverse quantization unit 53 performs inverse quantization on thequantization data decoded at the lossless decoding unit 52 with theformat corresponding to the quantization format of the quantization unit15 in FIG. 3. The inverse orthogonal transform unit 54 performs inverseorthogonal transform on the output of the inverse quantization unit 53with the format corresponding to the orthogonal transform format of theorthogonal transform unit 14 in FIG. 3 and outputs to the adding unit55.

The adding unit 55 adds the data after inverse orthogonal transform toprediction image data supplied from the selector 73, to generate decodedimage data and outputs to the deblocking filter 56 and intra predictionunit 71.

The deblocking filter 56 performs filtering processing as to the decodedimage data supplied from the adding unit 55, removes block distortionand then supplies to and stores at the frame memory 61, and outputs tothe screen rearranging buffer 57.

The screen rearranging buffer 57 performs rearranging of the images.That is, the order of the frame rearranged in order for encoding by thescreen rearranging buffer 12 of FIG. 3 is rearranged to the originalorder for display and is output to the D/A converting unit 58.

The D/A converting unit 58 performs D/A conversion on the image datasupplied from the screen rearranging buffer 57, so as to display theimage by outputting to an unshown display.

The quantization parameter calculating unit 59 restores quantizationparameters based on information supplied from the lossless decoding unit52, and outputs to the inverse quantization unit 53. FIG. 19 illustratesthe configuration of the quantization parameter calculating unit, withthe having a computing unit 591 and a quantization parameters memoryunit 592.

The computing unit 591 uses information supplied from the losslessdecoding unit 52 and quantization parameters stored in the quantizationparameters memory unit 592 to restore the quantization parameter used inthe quantization in the encoding to which the block to be decoded hasbeen subjected to, and outputs to the inverse quantization unit 53. Thecomputing unit 591 also stores the quantization parameter of the blockto be decoded in the quantization parameters memory unit 592.

The computing unit 591 uses, for example, the “pic_init_qp_minus26”extracted from the parameter set, and the “slice_qp_delta” extractedfrom the slice header, to perform the computation of Expression (30),calculates the quantization parameters SliceQPY, and outputs to theinverse quantization unit 53.

The computing unit 591 also uses the identification information anddifference information supplied from the lossless decoding unit 52 andthe quantization parameters of the decoded blocks stored in thequantization parameters memory unit 592, and calculates the quantizationparameter of the block to be decoded. The computing unit 591 outputs thecalculated quantization parameter to the inverse quantization unit 53.In this case, the computing unit 591 reads out, from the quantizationparameters of the decoded blocks stored in the quantization parametersmemory unit 592, the quantization parameters of decoded blocks spatiallyor temporally peripheral to the block to be decoded. The computing unit591 sets selection candidates in the same way as with the differencecomputing unit 192. For example, the computing unit 591 excludes atleast blocks where quantization parameters are redundant or blocks whereinverse quantization using quantization parameters is not performed, andtakes as selection candidates. Further, the computing unit 591 setsidentification information, i.e., index (ref_qp_block_index) equal tothe difference computing unit 192 as to the quantization parameters ofeach of the candidates. That is to say, the computing unit 591 sets theindex (ref_qp_block_index) with adjacent decoded blocks in apredetermined order of array. The computing unit 591 performscomputation of the Expression (31) using the quantization parameter“ref_qp(ref_qp_block_index)” corresponding to identification informationsupplied from the lossless decoding unit 52, i.e., the predictionquantization parameter and the difference indicated by the differenceinformation supplied from the lossless decoding unit 52 (qb_qp_delta).The computing nit 91 outputs the calculated quantization parameter(CurrentQP) to the inverse quantization unit 53 as the quantizationparameter to be decoded. Also, in the event that there is no selectioncandidate, the computing unit 591 outputs the quantization parameter ofthe initial value in the slice to the inverse quantization unit 53.

Also, in the event that information specifying the order of array ofblocks has been extracted from the stream information, the computingunit 591 sets index (ref_qp_block_index) with the decoded blocks in thespecified order of array. Accordingly, even if the order of array ischanged at the image encoding device 10, the quantization parametersused at the image encoding device 10 can be restored.

Returning to FIG. 18, the frame memory 61 holds the decoded image dataafter filtering processing supplied from the deblocking filter 24.

The intra prediction unit 71 generates prediction image data based onthe prediction mode information supplied from the lossless decoding unit52 and decoded image data supplied from the adding unit 55, and outputsthe generated prediction image data to the selector 73.

The motion compensation unit 72 reads out reference image data from theframe memory 61 based on the prediction mode information and motionvector supplied from the lossless decoding unit 52 and performs motioncompensation, to generate prediction image data. The motion compensationunit 72 outputs the generated prediction image data to the selector 73.Also, the motion compensation unit 72 generates prediction image datawhile switching filter properties in accordance with the magnitude ofthe motion vectors.

The selector 73 selects the intra prediction unit 71 in the case ofintra prediction and the motion compensation unit 72 in the case ofinter prediction, based on the prediction mode information supplied fromthe lossless decoding unit 52. The selector 73 outputs the predictionimage data generated at the selected intra prediction unit 71 or motioncompensation unit 72 to the adding unit 55.

The selector 73 selects the intra prediction unit 71 in the case ofintra prediction and the motion compensation unit 72 in the case ofinter prediction, based on the prediction mode information supplied fromthe lossless decoding unit 52. The selector 73 outputs the predictionimage data generated at the selected intra prediction unit 71 or motioncompensation unit 72 to the adding unit 55.

<5. Operation of Image Decoding Device>

Next, operation of the image decoding device 50 will be described withreference to the flowchart in FIG. 20.

In step ST91, the storage buffer 51 stores the stream information whichhas been supplied thereto. In step ST92, the lossless decoding unit 52performs lossless decoding processing. The lossless decoding unit 52decodes the stream information supplied from the storage buffer 51. Thatis, the quantization data of each picture encoded by the losslessencoding unit 16 in FIG. 3 is obtained. Also, the lossless decoding unit52, performs lossless encoding of prediction mode information includedin the stream information, and in the event that the obtained predictionmode information is information relating to the intra prediction mode,outputs the prediction mode information to the intra prediction unit 71.Also, in the event that the prediction mode information is informationrelating to the inter prediction mode, the lossless decoding unit 52outputs the prediction mode information to the motion compensation unit72. Further, the lossless decoding unit 52 outputs the difference motionvectors, threshold values, or threshold generating information, obtainedby decoding the stream information, to the motion compensation unit 72.

In step ST93, the inverse quantization unit 53 performs inversequantization processing. The inverse quantization unit 53 performsinverse quantization on the quantization data decoded by the inversedecoding unit 52 with properties corresponding to the properties of thequantization unit 15 in FIG. 3.

In step ST94, the inverse orthogonal transform unit 54 performs inverseorthogonal transform processing. The inverse orthogonal transform unit54 performs inverse orthogonal transform on the transform coefficientdata subjected to inverse quantization by the inverse quantization unit53 with properties corresponding to the properties of the orthogonaltransform unit 14 of FIG. 3.

In step ST95, the adding unit 55 generates the decoded image data. Theadding unit 55 adds the data obtained by being performed inverseorthogonal transform processing to the prediction image data selected inthe later-described step ST99, and generates decoded image data. Thus,the original image is decoded.

In step ST96, the deblocking filter 56 performs filtering processing.The deblocking filter 56 performs filtering processing of the decodedimage data output from the adding unit 55, and removes block distortionincluded in the decoded image.

In step ST97, the frame memory 61 performs storage processing of thedecoded image data. Note that decoded image data stored in the framememory 61 and decoded image data output from the adding unit 55 are usedas reference image data to generate prediction image data.

In step ST98, the intra prediction unit 71 and motion compensation unit72 perform prediction processing. The intra prediction unit 71 andmotion compensation unit 72 each perform prediction processingcorresponding to the prediction mode information supplied from thelossless decoding unit 52.

That is, when the prediction mode information of the intra prediction issupplied from the lossless decoding unit 52, the intra prediction unit71 performs intra prediction processing based on the prediction modeinformation and generates prediction image data. Also, in the event thatthe prediction mode information of the inter prediction is supplied fromthe lossless decoding unit 52, the motion compensation unit 72 performsmotion compensation based on the prediction mode information andgenerates prediction image data.

In step ST99, the selector 73 selects prediction image data. Theselector 73 selects the prediction image supplied from the intraprediction unit 71 and prediction image data supplied from the motioncompensation unit 72 and supplies the selected prediction image data tothe adding unit 55, so as to add to the output of the inverse orthogonaltransform unit 54 in step ST95, as described above.

In step ST100, the screen rearranging buffer 57 performs imagerearranging. That is to say, in the screen rearranging buffer 57, theorder of frames rearranged for encoding by the screen rearranging buffer12 of the image encoding device 10 in FIG. 3 is rearranged to theoriginal order for display.

In step ST101, the D/A conversion unit 58 performs D/A conversion on theimage data from the screen rearranging buffer 57. This image is outputto the unshown display and the image is displayed.

Next, the prediction image generating processing in step ST98 in FIG. 20will be described with reference to the flowchart in FIG. 21.

In step ST111, the lossless decoding unit 52 determines whether or notthe current block has been intra encoded. In the event of the predictionmode information obtained by performing lossless decoding being intraprediction mode information, the lossless decoding unit 52 supplies theprediction mode information to the intra prediction unit 63 and advancesto step ST112. Also, in the event of the prediction mode information notbeing inter prediction mode information, the lossless decoding unit 52supplies the prediction mode information to the motion compensation unit72, and proceeds to step ST113.

In step ST112, the intra prediction unit 71 performs intra predictionprocessing. The intra prediction unit 71 performs intra prediction usingdecoded image data before deblocking filter processing and predictionmode information supplied from the adding unit 55, and generatesprediction image data.

In step ST113, the motion compensation unit 72 performs inter predictionimage generating processing. The motion compensation unit 72 reads outreference image data from the frame memory 61 and generates predictionimage data, based on information supplied from the lossless decodingunit 52 such as prediction mode information and so forth.

FIG. 22 is a flowchart illustrating processing relating to quantizationparameters in decoding. In step ST121, the image decoding device 50extracts information to obtaining the quantization parameter unitminimum size. The image decoding device 50 extracts information forobtaining the quantization parameter unit minimum size, e.g., “log2_min_qp_unit_size_offset” from the stream information, and advances tostep ST122.

In step ST122, the image decoding unit 50 calculates the quantizationparameter unit minimum size. The image decoding unit 50 performs thecomputation of Expression (26) using “log 2_min_qp_unit_size_offset”,and parameter “log 2_min_transform_unit_size_minus2” which decides thetransform unit minimum size (MinTransformUnitSize), and calculates thequantization parameter unit minimum size (MinQpUnitSize). Also, theimage decoding nit 50 may calculate the quantization parameter unitminimum size (MinQpUnitSize) by the computation of Expression (29).

In step ST123, the image decoding unit 50 determines whether or notthere exists a frame to decode. In the event that there exists a frameto decode, the image decoding unit 50 advances to step ST124, and in theevent that there exists no frame to decode, ends the processing.

In step ST124, the image decoding unit 50 determines whether or notthere exists a slice to decode. In the event that there exists a sliceto decode, the image decoding unit 50 advances to step ST125, and in theevent that there exists no slice to decode, returns to step ST123.

In step ST125 the image decoding unit 50 extracts information forobtaining the quantization parameter of the initial value in the slice.The lossless decoding unit 52 of the image decoding unit 50 extracts,for example, “pic_init_qp_minus26” from a picture parameter set (PPS:picture parameter set). Also, “slice_qp_delta” is extracted from theslice header, and advances to step ST126.

In step ST126, the image decoding unit 50 calculates the quantizationparameter of the initial value in the slice. The quantization parametercalculating unit 59 of the image decoding unit 50 performs computationof Expression (30) using “pic_init_qp_minus26” and “slice_qp_delta”,calculates quantization parameter SliceQPY, and advances to step ST127.

In step ST127, the image decoding unit 50 determines whether or notthere exists a coding unit CU to decode. In the event that there existsa coding unit to decode, the image decoding unit 50 advances to stepST128, and in the event that there exists none, returns to step ST124.

In step ST128, the image decoding unit 50 sets identificationinformation to the selection candidates. The quantization parametercalculating unit 59 of the image decoding unit 50 sets identificationinformation to the selection candidates in the same way as with theinformation generating unit 19 of the image encoding device 10. That isto say, the quantization parameter calculating unit 59 takesquantization parameters of decoded codings spatially or temporallyperipheral to the coding unit to be decoded, as selection candidates.Also, in the event that no quantization parameters are set to the blockdue to being a skip block or having no residual information, or in theevent that a quantization parameter is equal to another candidate, theseare excluded from selection candidates. The quantization parametercalculating unit 59 sets identification information equal to the imageencoding device 10, e.g., index (ref_qp_block_index), to thequantization parameters of the candidates, and advances to step ST129.

In step ST129, the image decoding unit 50 obtains identificationinformation and difference information. The lossless decoding unit 52 ofthe image decoding unit 50 extracts the identification information anddifference information included in the stream information at the imageencoding device 10, i.e., the index (ref_qp_block_index) and differenceqb_qp_delta). The lossless decoding unit 52 supplies the extractedidentification information and difference information to thequantization parameter calculating unit 59 ad advances to step ST130.

In step ST130, the image decoding unit 50 uses the identificationinformation and difference information to calculate quantizationparameters. The quantization parameter calculating unit 59 of the imagedecoding unit 50 performs the computation of Expression (31) using thequantization parameter “ref_qp (ref_qp_block_index” corresponding to theindex (ref_qp_block_index) which is identification information, and(qb_qp_delta) which is difference information. That is to say, by addingthe difference to the prediction quantization parameter, thequantization parameter of the coding unit to be decoded is calculated.The quantization parameter calculating unit 59 outputs the quantizationparameter of the coding unit to be decoded (CurrentQP) to the inversequantization unit 53, and returns to step ST124.

Thus, by using identification information and difference informationincluded in the stream information, quantization parameters related tothe block to be decoded can be restored even if quantization parametersof each of the blocks are not included in the stream information. Thatis to say, even if the encoding efficiency of the quantizationparameters has been improved by using identification information anddifference information at the image encoding device 10, the quantizationparameters relating to each of the blocks can be restored and decodingprocessing can be correctly performed to generate a decoded image at theimage decoding unit 50.

<6. Other Operations of Image Encoding Device and Image Decoding Device>

With the above-described operations of the image encoding device andimage decoding device, quantization parameters of encoded blocksspatially or temporally adjacent to the block to be encoded are taken asselection candidates. Also, a quantization parameter selected from theselection candidates in accordance to a quantization parameter set as tothe block to be encoded is taken as a prediction quantization parameter.Further, encoding efficiency of quantization parameters is improved byincluding, in the stream information, identification information forselecting the prediction quantization parameter from the selectioncandidates, and difference information indicating the difference betweenthe prediction quantization parameter and the quantization parameter setto the block to be encoded.

However, the selection candidates are not restricted to quantizationparameters of encoded blocks spatially or temporally adjacent to theblock to be encoded, and the last updated quantization parameter may beincluded in the selection candidates. As described later, even a blockwhere encoded blocks spatially or temporally adjacent do not involveinverse quantization, a quantization parameter of a block at a positionnear to the block to be encoded can be set as a prediction quantizationparameter. Further, quantization parameters may be implicitly orexplicitly predicted quantization parameters of selection candidates,and difference information indicating the difference between thepredicted quantization parameters and the quantization parameter of theblock to be encoded, may be generated.

Next, description will be made regarding a case of deciding thequantization parameter unit minimum size (MinQpUnitSize) in accordancewith the coding unit size, and implicitly or explicitly selecting aprediction quantization parameter from quantization parameters ofselection candidates, as another operation of the image encoding deviceand image decoding device. Note that description will be made belowregarding to portions differing from the image encoding device and imagedecoding device described above.

In the event of implicitly or explicitly selecting a predictionquantization parameter from quantization parameters of selectioncandidates, the image encoding device includes distinguishinginformation “qp_explicit_flag” indicating whether to explicitly orimplicitly decide quantization parameters. Also, an arrangement may bemade with the image encoding device and image decoding device wherewhether to implicitly decide or explicitly decide quantizationparameters is decoded beforehand.

To implicitly decide quantization parameters means a predictionquantization parameter equal to the image encoding device can beselected at the image decoding device, without supplying identificationinformation to select the prediction quantization parameters from theselection candidates from the image encoding device to the imagedecoding device. Specifically, there is a method of selecting aquantization parameter from the selection candidates based on a priorityorder decided beforehand and deciding the prediction quantizationparameter, a method of taking a stochastic value of quantizationparameters of the selection candidates as a prediction quantizationparameter, a method of weighting quantization parameters of theselection candidates in accordance to distance from the current block,and taking a stochastic value of weighted quantization parameters as aprediction quantization parameter, or the like.

To explicitly decide quantization parameters means a predictionquantization parameter equal to the image encoding device can beselected at the image decoding device, by supplying identificationinformation to select the prediction quantization parameters from theselection candidates from the image encoding device to the imagedecoding device. Specifically, there is a method of calculating indexinformation specifying a selection candidate at the image encodingdevice and including this in the stream information, and using thequantization parameter of the selection candidate indicated in the indexinformation as the prediction quantization parameter at the imagedecoding device, a method where index information is not included inblocks regarding which quantization is not performed, and so forth.

FIG. 23 is a flowchart for describing another operation of the imageencoding device, illustrating slice encoding processing. In step ST141,the image encoding device 10 determines whether or not a coding unit CUto be encoded exists. In the event that a coding unit which has not beensubjected to encoding processing exists in a slice to be encodingprocessed, the image encoding device 10 advances to step ST142. In theevent that encoding processing has been completed for all coding unitsin the slice, the image encoding device 10 ends the slice encodingprocessing.

In step ST142, the image encoding device 10 splits the coding unit CU.The image encoding device 10 splits the coding unit CU as illustrated inFIG. 5, decides the size of the coding unit where the cost functionvalue is small, and advances to step ST143. Also, in order to enabledetermination of the size of the coding unit where the cost functionvalue is small, the image encoding device 10 includes in the streaminformation, for example, “Coding tree syntax”, a“split_coding_unit_flag” equivalent to the split flag in FIG. 5.

In step ST143, the image encoding device 10 determines whether or notinverse quantization is involved with the coding unit to be encoded. Inthe event that the coding unit CU to be encoded is a block of a modewhich does not need inverse quantization using quantization parametersto perform decoding, e.g., skip mode or I_PCM mode, or direct mode (CBP(Coded Block Pattern)=0) block, the image encoding device 10 returns tostep ST141, and in the event of a block where inverse quantization isperformed, advances to step ST144.

In step ST144, the image encoding device 10 determines whether or notthe size of the coding unit CU is “log 2MinQpUinitSize” or greater. Inthe event that the size of the coding unit CU is “log 2MinQpUnitSize” orgreater, the image encoding device 10 advances to step ST145. Also, inthe event that the size of the coding unit CU is not “log2MinQpUnitSize” or greater, the image encoding device 10 advances tostep ST152.

In step ST145 the image encoding device 10 decides a quantizationparameter QP for the coding unit CU to be encoded. The rate control unit18 of the image encoding device 10 decides the quantization parameter inaccordance with the complexity of the image of the coding unit asdescribed above, or such that the cost function value is small, andadvances to step ST146.

In step ST146, the image encoding device 10 determines whether or notdistinguishing information “qp_explicit_flag” enabling identification ofwhether quantization parameters are to be predicted implicitly orexplicitly is “1”. In the event that the distinguishing information“qp_explicit_flag” is “1”, and quantization parameters are to bepredicted explicitly, the image encoding device 10 advances to stepST147. Also, in the event that the distinguishing information“qp_explicit_flag” is “0” and quantization parameters are to bepredicted implicitly, the image encoding device 10 advances to stepST149. The image encoding device 10 compares the cost function value inthe event that the distinguishing information “qp_explicit_flag” is setto “1” and the cost function value in the event that the distinguishinginformation “qp_explicit_flag” is set to “0”, for example. The imageencoding device 10 sets the value of the distinguishing information“qp_explicit_flag” such that the encoding efficiency is higher based onthe comparison results. Also, in the event that the distinguishinginformation “qp_explicit_flag” can be set by the user, the imageencoding device 10 sets the distinguishing information“qp_explicit_flag” in accordance with user instructions.

In step ST147, the image encoding device 10 generates identificationinformation. The image encoding device 10 selects a candidate from theselection candidates such that the difference as to the quantizationparameter of the coding unit to be encoded is smallest at theinformation generating unit 19 as described above, and takes this to bea prediction quantization parameter. The image encoding device 10 takes,for example, quantization parameters of encoded blocks spatially ortemporally adjacent to the block to be encoded, the last updatedquantization parameter, and the processing procedures set at the headblock of the slice, as selection candidates. The image encoding device10 selects a candidate from the selection candidates where thedifference as to the quantization parameter of the coding unit to beencoded is smallest, and takes this as a prediction quantizationparameter. Further, the information generating unit 19 takes the index(ref_qp_block_index) of the selected candidate to be identificationinformation for selecting a prediction quantization parameter from theselection candidates, and advances to step ST148.

In step ST148, the image encoding device 10 includes the identificationinformation in the stream information. The image encoding device 10includes the identification information generated in step ST147, andadvances to step ST150.

En step ST149, the image encoding device 10 implicitly decides aprediction quantization parameter dQP. That is to say, the imageencoding device 10 predicts a quantization parameter with a method equalto the image decoding unit 50. As for a method for predicting thequantization parameter, the prediction quantization parameters isdecoded based on a priority order decided beforehand, for example. Also,a stochastic value of multiple candidate quantization parameters may betaken as a prediction quantization parameter. Further, a method ofweighting quantization parameters of the selection candidates inaccordance with distance from the current block, and taking a stochasticvalue of weighted quantization parameters as a prediction quantizationparameter, or the like, may be used. The image encoding device 10calculates the prediction quantization parameter and advances to stepST150.

In step ST150 the image encoding device 10 generates differenceinformation. The image encoding device 10 calculates the differencebetween the prediction quantization parameter indicated by theidentification information generated in step ST147 and the quantizationparameter decided in step ST145, or the difference between theprediction quantization parameter decided in step ST149 and thequantization parameter decided in step ST145. The image encoding device10 generates difference information indicating the calculated differenceand advances to step ST151.

In step ST151, the image encoding device 10 includes the differenceinformation and distinguishing information in the stream information.The image encoding device 10 includes the difference informationgenerated in step ST151 and the distinguishing information“qp_explicit_flag” used in step ST146 in the stream information. Theimage encoding device 10 includes the distinguishing information in oneof, for example, the sequence parameter set, picture parameter set,slice header, or the like, and advances to step ST152.

In step ST152, the image encoding device 10 performs quantization of thecoding unit CU. The image encoding device 10 performs quantization ofthe coding unit using the decided quantization parameter, and returns tostep ST141.

FIG. 24 is an operation example in a case of implicitly predictingquantization parameters, and FIG. 25 illustrates a flowchart example ina case of explicitly predicting quantization parameters. Note that acase of three selection candidates is illustrated to facilitatedescription.

As illustrated in (A) in FIG. 24, the quantization parameter of theblock to be encoded in the frame c be encoded is, for example, “QP_0”.Also, the three candidates are the quantization parameter “QP_A” of theencoded block adjacent to the left, the quantization parameter “QP_B” ofthe adjacent encoded block, and the quantization parameter “QP_LS” of adecoded coding unit.

In FIG. 25, in step ST161 the image encoding device 10 determineswhether or not quantization parameters “QP_A” “QP_B” can be referenced.In the event that the encoded block adjacent to the left and encodedblock adjacent above are not blocks of a mode which does not needinverse quantization using quantization parameters to perform decoding,e.g., skip mode or I_PCM mode, or direct mode (CBP (Coded BlockPattern)=0) block, the image encoding device 10 determines thatreference can be made and advances to step ST162. Also, in the eventthat at least one of the quantization parameter “QP_A” and thequantization parameter “QP_B” is a mode which does not need inversequantization, advances to step ST163.

In step ST162, the image encoding device 10 takes the average value ofquantization parameters “QP_A” “QP_B” to be prediction quantizationparameter dQP. That is to say, as illustrated in (B) in FIG. 24, in theevent that quantization parameters “QP_A” “QP_B” can be referenced, theaverage value of the quantization parameters “QP_A” “QP_B”“(QP_A+QP_B+1)/2” is taken as the prediction quantization parameter dQP.

In step ST163, the image encoding device 10 determines whether or notthe quantization parameter “QP_A” can be referenced. In the event thatthe encoded block adjacent to the left is not a mode where there is noneed to perform inverse quantization, the image encoding device 10determines that this can be referenced, and advances to step ST164.Also, in the event that the encoded block adjacent to the left is a modewhere there is no need to perform inverse quantization, the imageencoding device 10 determines that this cannot be referenced, andadvances to step ST165.

In step ST164, the image encoding device 10 takes the quantizationparameter “QP_A” as the prediction quantization parameter dQP. That isto say, in the event that the quantization parameter “QP_A” can bereferenced and the quantization parameter “QP_B” cannot be referenced,as illustrated in (C) in FIG. 24, the quantization parameter “QP_A” istaken as the prediction quantization parameter dQP. Note that in FIG. 24and the later-described FIG. 26, blocks of a mode which do not needinverse quantization, i.e., blocks that cannot be referenced, areindicated by hatching.

In step ST165, the image encoding device 10 determines whether or notthe quantization parameter “QP_B” can be referenced. In the event thatthe encoded block adjacent above is not a mode where there is no need toperform inverse quantization, the image encoding device 10 determinesthat this can be referenced, and advances to step ST166. Also, in theevent that the encoded block adjacent above is a mode where there is noneed to perform inverse quantization, the image encoding device 10determines that this cannot be referenced, and advances to step ST167.

In step ST166, the image encoding device 10 takes the quantizationparameter “QP_B” as the prediction quantization parameter dQP. That isto say, in the event that the quantization parameter “QP_B” can bereferenced and the quantization parameter “QP_A” cannot be referenced,as illustrated in (D) in FIG. 24, the quantization parameter “QP_B” istaken as the prediction quantization parameter dQP.

In step ST167, the image encoding device 10 takes the quantizationparameter “QP_LS” as the prediction quantization parameter dQP. Asillustrated in (E) in FIG. 24, in the event that the encoded blockadjacent to the left and the encoded block adjacent above are a modewhere there is no need to perform inverse quantization, the quantizationparameter “QP_LS” is taken as the prediction quantization parameter dQP.

FIG. 26 illustrates another operation example of a case of implicitlypredicting quantization parameters. For example, as illustrated in (E)in FIG. 24, in the event that the encoded block adjacent to the left andthe encoded block adjacent above are a mode where there is no need toperform inverse quantization, the predetermined quantization parametermay be generated by increasing the number of selection candidates. Forexample, as illustrated in (A) in FIG. 26, the quantization parameter“QP_C” of the encoded block adjacent to the upper right, thequantization parameter “QP_D” of the encoded block adjacent to the upperleft, and the quantization parameter “QP_E” of the encoded blockadjacent to the lower left, are added to the selection candidates.

In the event that the quantization parameters “QP_C”, “QP_D”, and “QP_E”can be referenced as illustrated in (B) in FIG. 26, the image encodingdevice 10 takes the average value “(QP_C+QP_D+1)/2” of the quantizationparameters “QP_C” and “QP_D”, or the median, as the predictionquantization parameter dQP.

In the event that the quantization parameters “QP_C” and “QP_D” can bereferenced as illustrated in (C) in FIG. 26, the image encoding device10 takes the average value “(QP_C+QP_D+1)/2” of the quantizationparameters “QP_C” and “QP_D” as the prediction quantization parameterdQP.

In the event that the quantization parameters “QP_D” and “QP_E” can bereferenced as illustrated in (D) in FIG. 26, the image encoding device10 takes the average value “(QP_)+QP_E+1)/2” of the quantizationparameters “QP_D” and “QP_E” as the prediction quantization parameterdQP.

In the event that the quantization parameters “QP_C” and “QP_E” can bereferenced as illustrated in (E) in FIG. 26, the image encoding device10 takes the average value “(QP_C+QP_E+1)/2” of the quantizationparameters “QP_C” and “QP_E” as the prediction quantization parameterdQP.

In the event that the quantization parameters “QP_C”, “QP_D”, and “QP_E”cannot be referenced as illustrated in (F) in FIG. 26, the imageencoding device 10 takes the quantization parameter “QP_LS” as theprediction quantization parameter dQP. Note that FIG. 27 illustrates aprogram for performing the operations of (B) through (D) in FIG. 24 and(B) through (F) in FIG. 26.

Also, in the event that the number of quantization parameters that canreferenced is one, as illustrated in (G) through (I) in FIG. 26, thisquantization parameter that can referenced may be used as the predictionquantization parameter dQP.

Thus, the image encoding device 10 takes quantization parameters such asencoded blocks spatially or temporally adjacent to a block to be encodedas selection candidates, and selects a prediction quantization parameterfrom the selection candidates in accordance with a set quantizationparameter. Also, the image encoding device 10 generates identificationinformation for selecting a prediction quantization parameter from theselection candidates. Further, the image encoding device 10 generatesdifference information indicating difference between the predictionquantization parameter and the quantization parameter set to the blockto be encoded. The image encoding device 10 includes the generatedidentification information and difference information in streaminformation. By performing such processing, the difference between thequantization parameter of the block to be encoded and the predictionquantization parameter can be prevented from becoming a great value.Accordingly, the image encoding device 10 can improve encodingefficiency of quantization parameters.

Also, in the event of implicitly prediction quantization parameters, aprediction quantization parameter equal to the image encoding device 10can be used at the image decoding device 50, without includingidentification information to select prediction quantization parametersfrom the selection candidates in the stream information. Further, byincluding distinguishing information in the stream information, explicitprediction of prediction quantization parameters and implicit predictionof prediction quantization parameters can be adaptively switched.

FIG. 28 is a flowchart for describing other operations of the imagedecoding device. In step ST127 in FIG. 22, in the event thatdetermination is made that a coding unit to decode exists, the imagedecoding unit 50 performs processing from step ST171, and performsdecoding of the coding unit.

In step ST171, the image decoding unit 50 extracts information. Theimage decoding unit 50 extracts information from the stream informationto use in decoding of the coding unit. For example, information “Codingtree syntax” enabling determination of the size of the coding unit,information “log 2_min_qp_unit_size_offset” enabling determination ofthe quantization parameter unit minimum size, distinguishing information“qp_explicit_flag”, and so forth, are extracted, and advances to stepST172.

In step ST172, the image decoding unit 50 splits the coding unit CU. Theimage decoding unit 50 splits the coding unit CU based on“split_coding_unit_flag” and so forth included in the streaminformation, and advances to step ST173.

In step ST173, the image decoding unit 50 determines whether or not thecoding unit CU to be decoded involves inverse quantization. In the eventthat the coding unit CU to be encoded is a mode where inversequantization using quantization parameters is performed, the imagedecoding unit 50 advances to step ST174, and in the event of a blockwhere inverse quantization using quantization parameter is notnecessary, the decoding processing ends.

In step ST174, the image decoding unit 50 determines whether or not thesize of the coding unit CU is “log 2MinQpUnitSize” or greater. In theevent that the size of the coding unit CU is “log 2MinQpUnitSize” orgreater, the image decoding unit 50 advances to step ST175. Also, in theevent that the size of the coding unit CU is not “log 2MinQpUnitSize” orgreater, the image decoding unit 50 advances to step ST180.

In step ST175, the image decoding unit 50 determines whether or notdistinguishing information “qp_explicit_flag” is “1”. In the event thatthe distinguishing information “qp_explicit_flag” included in the streaminformation is “1”, and quantization parameters are to be predictedexplicitly, the image decoding unit 50 advances to step ST176. Also, inthe event that the distinguishing information “qp_explicit_flag” is “0”and quantization parameters are to be predicted implicitly, the imagedecoding unit 50 advances to step ST178.

In step ST176, the image decoding unit 50 extracts the index(ref_qp_block_index) from the stream information and advances to stepST177.

In step ST177, the image decoding unit 50 decides the predictionquantization parameter dQP. The image decoding unit 50 selects thequantization parameter based on the index (ref_qp_block_index) fromquantization parameters of selection candidates, equal to the imageencoding device 10, decides the selected quantization parameter to bethe prediction quantization parameter dQP, and advances to step ST179.

In step ST178, the image decoding unit 50 implicitly decides theprediction quantization parameter dQP. The image decoding unit 50predicts the quantization parameter with a method equal to the imageencoding device 10. As for a method for predicting the quantizationparameter, a quantization parameter may be decided based on an order ofpriority decided beforehand, for example. Also, a stochastic value ofquantization parameters of the selection candidates may be taken as aprediction quantization parameter. Further, a method of weightingquantization parameters of the selection candidates in accordance withdistance from the current block, and taking a stochastic value ofweighted quantization parameters as a prediction quantization parameter,or the like, may be used. The image decoding unit 50 predicts thequantization parameter and advances to step ST179.

In step ST179, the image decoding unit 50 calculates the quantizationparameter QP of the current coding unit CU. The image decoding unit 50obtains difference information “qb_qp_delta” from the streaminformation, adds this difference information to the predictionquantization parameter dQP, calculates the quantization parameter of thecoding unit to be decoded, and advances to step ST180.

In step ST180, the image decoding unit 50 performs inverse quantizationof the coding unit. The image decoding unit 50 performs inversequantization of the coding unit using the decoded quantizationparameter.

Accordingly, the image decoding unit 50 can perform decoding of imagesusing quantization parameters equal to the quantization parameters usedby the image encoding device.

<7. Case of Software Processing>

The series of the processing described above may be performed byhardware, software, or a combined configuration of both. In the casewhere processing by software is performed, a program in which isrecorded a processing sequence is installed in memory within a computerbuilt into dedicated hardware, and is executed. Alternatively, a programmay be installed in a general-purpose computer by which various types ofprocessing can be performed.

FIG. 29 is a diagram exemplifying a schematic configuration of acomputer device executing the above-described series of processing by aprogram. A CPU 80 of the computer device 80 executes various types ofprocessing according to a program stored in ROM 802 or recorded in arecording unit 808.

Programs which the CPU 801 executes, data, and so forth, are stored inthe RAM 803 as appropriate. The CPU 801, ROM 802, and RAM 803 aremutually connected via a bus 804.

An input/output interface 805 is also connected to the CPU 801 via thebus 804. An input unit 806 such as a touch panel, keyboard, mouse,microphone, or the like, and an output unit 807 made up of a display orthe like, are connected to the CPU 801 as well. The CPU 801 executesvarious types of processing in accordance with commands input from theinput unit 806. The CPU 801 then outputs the results of processing tothe output unit 807.

The recording unit 808 connected to the input/output interface 805 inmade up of a hard disk for example, and records programs which the CPU801 executes, and various types of data. A communication unit 809communicates with external devices via cable or wireless communicationmedia such as networks like the Internet or local area networks, ordigital broadcasting or the like. Also, the computer device 80 mayacquire programs via the communication unit 809 and record in the ROM802 or recording unit 808.

In the event that removable media 85 such as a magnetic disk, opticaldisc, magneto-optical disk, or semiconductor memory or the like ismounted to the drive 810, these are driven, and programs, data, and thelike, recorded therein, are obtained. The obtained programs and data aretransferred to the ROM 802 or RAM 803 or recorded unit 808 as necessary.

The CPU 801 reads out and executes a program performing theabove-described series of processing, and performs encoding processingof image signals recorded in the recording unit 808 or removable media85 or imaging signals supplied via the communication unit 809, ordecoding processing of stream information.

<8. Case of Applying to Electronic Devices>

Also, in the above, the H.264/AVC format has used as the encodingformat/decoding format, but the present technology can be also appliedto the image encoding device/image decoding device which uses anencoding format/decoding format which performs other motionprediction/compensation processing.

Furthermore, the present technology can be applied to the image encodingdevice and image decoding device used at the time of receiving streaminformation obtained by performing encoding processing, as with MPEG,H.26x or the like, via network media such as satellite broadcasting,cable TV (television), the Internet, cellular telephone, or the like, orat the time of processing on a storage medium such as an optical disc ormagnetic disk, and flash memory.

Next, description will be made regarding an electronic device to whichthe above-described image encoding device 10 and image decoding device50 have been applied.

FIG. 30 exemplarily illustrates a schematic configuration of atelevision apparatus to which the present technology has been applied.The television apparatus 90 has an antenna 901, a tuner 902, ademultiplexer 903, a decoder 904, a video signal processing unit 905, adisplay unit 906, an audio signal processing unit 907, a speaker 908,and an external interface unit 909. Furthermore, the televisionapparatus 90 has a control unit 910, a user interface unit 911 or thelike.

The tuner 902 performs demodulation by choosing a desired channel fromthe broadcast wave signals received at the antenna 901, and outputs theobtained stream to the demultiplexer 903.

The demultiplexer 903 extracts the packet of a video and audio of aprogram to be viewed from a stream and outputs the data of extractedpackets to the decoder 904. Also, the demultiplexer 903 supplies thepackets of data such as EPG (Electronic Program Guide) to the controlunit 910. Note that in the event that scrambling has been performed,descrambling is performed at a demultiplexer or the like.

The decoder 904 performs decoding process of the packet, and outputs thevideo data generated by being subjected to decoding processing to thevideo signal processing unit 905 and audio data to the audio signalprocessing unit 907.

The video signal processing unit 905 performs video processing accordingto the noise reduction and user settings, on the video data. The videosignal processing unit 905 generates video data for displaying programson the display unit 906 and image data according to processing based onapplications supplied through the network. Also, the video signalprocessing unit 905 generates video data to display menu screens or thelike such as for selection of items, and superimposes this on video dataof the program. The video signal processing unit 905 generates drivingsignals based on the video data generated in this way and drives thedisplay unit 906.

The display unit 906 drives a display device (e.g., liquid crystaldisplay device or the like) based on the driving signal from the videosignal processing unit 905 so as to display the video of the program.

The audio signal processing unit 907 subjects the audio data topredetermined processing such as noise reduction and performs audiooutput by performing D/A converting processing and amplifying processingof the audio data after processing and supplying to the speaker 908.

The external interface unit 909 is an interface to be connected toexternal equipment or a network, and performs data transmission andreception of such as video data or audio data.

The user interface unit 911 is connected to the control unit 910. Theuser interface unit 911 is configured of an operation switch or a remotecontrol signal receiver or the like, and supplies operation signalsaccording to user operation to the control unit 910.

The control unit 910 is configured using a CPU (Central ProcessingUnit), memory or the like. The memory stores programs to be executed bythe CPU, and various data necessary for the CPU to perform processing,EPG data, data obtained via a network, and the like. The program storedin the memory is read out by the CPU at a predetermined timing such asat the time of starting up the television apparatus 90 and is executed.The CPU controls each part so that the television apparatus 90 operatesaccording to user operations by executing a program.

Note that with the television apparatus 90, a bus 912 is provided toconnect a tuner 902, a demultiplexer 903, a video signal processing unit905, an audio signal processing unit 907, an external interface unit 909and a control unit 910.

With the television apparatus thus configured, the function of the imagedecoding device (image decoding method) of the present application isprovided to the decoder 904. Therefore, even if processing is performedin the image encoding processing at the broadcasting station side toreduce the amount of code necessary for transmitting quantizationparameters, the television device can correctly restore the quantizationparameters and generate a decoded image.

FIG. 31 exemplarily illustrates a schematic configuration of thecellular telephone to which the present technology has been applied. Thecellular telephone 92 has a communication unit 922, an audio codec 923,a camera unit 926, an image processing unit 927, a multiplex separatingunit 928, a record reproduction unit 929, a display unit 930, and acontrol unit 931. These are connected each other through a bus 933.

Also, an antenna 921 is connected to the communication unit 922, and aspeaker 924 and microphone 925 are connected to the audio codec 923.Furthermore, an operating unit 932 is connected to the control unit 931.

The cellular telephone 92 performs various operation such astransmission and reception of audio signals, transmission and receptionof email and image data, image shooting, data recording, and so forth,in various modes such as audio call mode or data communication mode.

In an audio call mode, audio signals generated at the microphone 925 areconverted to audio data and data compression at the audio codec 923 andsupplied to the communication unit 922. The communication unit 922performs demodulation processing of the audio data and frequencyconversion processing of audio data to generate transmission signals.Also, the communication unit 922 supplies transmission signals to theantenna 921 so as to be transmitted to an unshown base station. Also,the communications unit 922 performs amplification, frequency conversionprocessing, and demodulation processing of reception signals received atthe antenna 921, and supplies the obtained audio data to the audio codec923. The audio codec 923 performs data decompression of audio data andconversion to the analog audio signals and outputs to the speaker 924.

Also, in data communication mode, in the event of performing emailtransmission, the control unit 931 receives text data input by theoperation of the operating unit 932 and displays the input text to thedisplay unit 930. Also, the control unit 931 generates email data basedon user instructions at the operating unit 932 and supplies to thecommunication unit 922. The communication unit 922 performs modulationprocessing, frequency conversion processing, and so forth of the emaildata, and transmits the obtained transmission signals from the antenna921. Also, the communication unit 922 performs amplification, frequencyconversion processing, and demodulation processing of the receptionsignals received with the antenna 921, and restores the email data. Thisemail data is supplied to the display unit 930 to display the contentsof the email.

Note that the cellular telephone 92 may store the received email data instorage medium in the recording/playback unit 929. The storage medium isany storage medium which is readable/writeable. For example, the storagemedium is semiconductor memory such as PAM or built-in flash memory,removable media such as a hard disk, a magnetic disk, an MO disc, anoptical disc, USB memory, a memory card, or the like.

In the event that image data is transmitted in a data communicationmode, the image data generated at the camera unit 926 is supplied to theimage processing unit 927. The image processing unit 927 performsencoding processing of the image data and generates stream information.

The multiplex separating unit 928 multiplexes stream informationgenerated at the image processing unit 927 and audio data supplied fromthe audio codec 923 by a predetermined format and supplies to thecommunication unit 922. The communication unit 922 performs demodulationprocessing, frequency conversion processing, and the like of themultiplexed data, and transmits the obtained transmission signals fromthe antenna 921. Also, the communication unit 922 performsamplification, frequency conversion processing, demodulation processing,or the like of the reception signals received at the antenna 921, andrestores the multiplexed data. This multiplexed data is supplied to themultiplex separating unit 928. The multiplex separating unit 928performs separating of the multiplexed data, and supplies the streaminformation to the image processing unit 927 and the audio data to theaudio codec 923.

The image processing unit 927 performs decoding processing of theencoded data, and generates image data. This image data is supplied tothe display unit 930 to display the received image. The audio codec 923converts the audio data into analog audio signals and supplies to thespeaker 924 to output the received audio.

With the cellular telephone device thus configured, the image processingunit 927 has functions of the present application. Accordingly, data canbe reduced when performing encoding processing and transmission ofimages, for example. Also, in the decoding processing of the receivedimage, the quantization parameters can be restored and a decoded imagecan be generated.

FIG. 32 exemplarily illustrates a schematic configuration of therecording playback device to which the present technology has beenapplied. The recording/playback device 94 records, for example, audiodata and video data of the received broadcast program to a recordingmedium, and provides the recorded data to a user in a timing accordingto the instructions of the user. Also, an arrangement can be made suchthat the recording/playback device 94 may acquire, for example, audiodata and video data from other devices, so as to record to a recordingmedium. Furthermore, an arrangement can be made such that therecording/playback device 94 may perform, by decoding audio data andvideo data recorded in a recording medium to output image display andaudio output on monitor devices.

The recording/playback device 94 has a tuner 941, an external interfaceunit 942, an encoder 943, an HDD (Hard Disk Drive) unit 944, a diskdrive 945, a selector 946, a decoder 947, an OSD (On-Screen Display)unit 948, a control unit 949, and a user interface unit 950.

The tuner 941 chooses a station of a desired channel from the broadcastsignals received at an unshown antenna. The tuner 941 outputs theencoded stream obtained by demodulating the reception signals of thedesired channel to the selector 946.

The external interface unit 942 is configured of at least any one of anIEEE1394 interface, a network interface unit, a USB interface, a flashmemory interface, and so forth. The external interface unit 942 is aninterface to be connected to an external device, network, memory card,or the like, and performs data reception of such as video data and audiodata to record.

The encoder 943 performs encoding processing in a case where video dataand audio data supplied from the external interface unit 942 are notencoded by a predetermined format and outputs stream information to theselector 946.

The HDD unit 944 records the content data such as the video or theaudio, various programs, other data, or the like, in a built-in harddisk, and also reads out these at the time of playback from the harddisk.

The disk drive 945 performs recording or playback of the signals to amounted optical disc. An optical disc is, e.g., a DVD disc (DVD-Video,DVD-RAM, DVD-R, DVD-RW, DVD+R, DVD+RW or the like) or a Blu-ray disk orthe like.

The selector 946 selects, at the time of the recording of video andaudio, either stream from the tuner 941 or encoder 943, and supplies toeither of HDD unit 944 and disk drive 945. Also, the selector 946supplies, at the time of the playback of video and audio, a streamoutput from the HDD unit 944 or disk drive 945 to the decoder 947.

The decoder 947 performs decoding process of the stream. The decoder 947supplies the generated video data to the OSD unit 948 by performingdecoding processing. Also, the decoder 947 outputs the generated audiodata by performing decoding processing.

The OSD unit 948 generates video data to display menu screens or thelike such as for the selection of items and superimposes this on thevideo data output from the decoder 947, and outputs.

The user interface unit 950 is connected to the control unit 949. Theuser interface unit 950 is configured of an operation switch or a remotecontrol signal receiver or the like and supplies operation signalsaccording to user operations to the control unit 949.

The control unit 949 is configured using a CPU or memory. The memorystores a program executed by CPU and necessary various data when the CPUperforming processing. The program stored in the memory is read out andexecuted at a predetermined timing such as at the time of start of therecording/playback device 94, by the CPU. The CPU controls each part sothat the recording/playback device 94 operates in accordance with useroperation, by executing a program.

With the recording/playback device thus configured, functions of thepresent application are provided to the encoder 943. Therefore, dataamount can be reduced when performing encoding processing and recordingof images, for example. Also, in the decoding processing of the recordedimage, the quantization parameters can be restored and a decoded imagecan be generated.

FIG. 33 exemplarily illustrates a schematic configuration of an imagingapparatus to which the present invention has been applied. The imagingapparatus 96 images a subject so as to display the image of the subjecton a display unit, and record this in a recording medium as image data.

The imaging apparatus 96 has an optical block 961, an imaging unit 962,a camera signal processing unit 963, an image data processing unit 964,a display unit 965, an external interface unit 966, a memory unit 967, amedia drive 968, an OSD unit 969, and a control unit 970. Also, a userinterface unit 971 is connected to the control unit 970. Furthermore,the image data processing unit 964 and external interface unit 966,memory unit 967, media drive 968, OSD unit 969, and control unit 970 andso forth are connected via a bus 972.

The optical block 961 is configured of a focusing lens, diaphragmmechanism, and so forth. The optical block 961 images an optical imageof a subject on an imaging face of the imaging unit 962. The imagingunit 962 is configured using a CCD or CMOS image sensor, and electricalsignals corresponding to the optical image are generated byphotoelectric conversion and supplied to the camera signal processingunit 963.

The camera signal processing unit 963 performs various kinds of camerasignal processing such as KNEE correction and gamma correction, colorcorrection and the like, to the electrical signals supplied from theimaging unit 962. The camera signal processing unit 963 supplies theimage data after camera signal processing to the image data processingunit 964.

The image data processing unit 964 performs encoding processing of theimage data supplied from the camera signal processing unit 963. Theimage data processing unit 964 supplies the stream information generatedby performing encoding processing to the external interface unit 966 andmedia drive 968. Also, the image data processing unit 964 performsdecoding processing of the stream information supplied from the externalinterface unit 966 and media drive 968. The image data processing unit964 supplies the generated image data to the display unit 965 byperforming decoding processing. Also, the image data processing unit 964performs processing to supply the image data supplied from the camerasignal processing unit 963 to the display unit 965, and processing tosuperimpose data for display acquired from the OSD unit 969 onto theimage data and supply to the display unit 965.

The OS) unit 969 generates data for display such as a menu screen or theicon made of signs, text or shapes, and outputs to the image dataprocessing unit 964.

For example, the external interface unit 966 is configured of USB inputand output terminals, and in a case of performing printing of the image,is connected to a printer. Also, a drive is connected to the externalinterface unit 966 according to need, and removable media such as amagnetic disk, optical disc, or the like is mounted as appropriate, anda program read out therefrom is installed according to need.Furthermore, the external interface unit 966 has a network interfaceconnected to a predetermined network such as a LAN or the Internet. Forexample, according to the instructions from the user interface unit 971,the control unit 970 reads out stream information from the memory unit967 so as to be supplied to the other devices connected via a networkfrom the external interface unit 966. Also, the control unit 970acquires stream information and image data supplied from other devicesvia a network, through the external interface unit 966, so as to supplythis to the image data processing unit 964.

For example, as for a recording medium driven by the media drive 968,any removable media which is readable and writable may be used, such asa magnetic disk, an MO disk, an optical disc, and semiconductor memory.Also, with a recording medium, the kind of removable media is alsooptional, and may be a tape device, may be a disk or may be a memorycard. As a matter of course this may be a non-contact IC card or thelike.

Also, an arrangement may be made where the media drive 968 and recordingmedium are integrated and, for example, are configured of a non-portablestorage medium such as a built-in type hard disk drive or SSD (SolidState Drive) or the like.

The control unit 970 is configured using a CPU memory, and so forth. Thememory stores programs to be stored by the CPU, and various types ofdata necessary for the CPU to perform processing. Programs stored in thememory are read out at predetermined timing such as at the time ofstartup of the imaging device 96 by the CPU, and are executed. The CPUcontrols each part so that operations of the imaging device 96correspond to user operations, by executing the program.

With the imaging device thus configured, the image data processing unit964 is provided with functions of the present application. Therefore, atthe time of encoding and recording the imaged image to the memory unit967 or a recording medium, the amount of data to be recorded can bereduced. Also, in the decoding processing of the recorded image, thequantization parameters can be restored and a decoded image can begenerated.

Furthermore, the present technology is not to be interpreted as beingrestricted to the above-described embodiments. The embodiments aredisclosed exemplarily, and it is clearly understood that one skilled inthe art can accomplish modifications and a substitutions of theembodiments without departing from the essence of the presenttechnology. That is, the Claims should be taken into consideration todetermine the essence of the present technology.

Also, the image decoding device and image encoding device according tothe present technology may assume the following configurations.

(1) An image decoding device, including:

an information acquiring unit configured to take quantization parametersof decoded blocks spatially or temporally adjacent to a block to bedecoded, as selection candidates, and extract, from stream information,difference information indicating difference as to a predictionquantization parameter selected from the selection candidates; and

a quantization parameter calculating unit configured to calculate, fromthe prediction quantization parameter and the difference information, aquantization parameter of the block to be decoded.

(2) The image decoding device according to (1), wherein the quantizationparameter calculating unit sets to the prediction quantization parametera quantization parameter in an order indicated by identificationinformation included in the stream information, with the adjacentdecoded blocks in a predetermined order.

(3) The image decoding device according to (1), wherein the quantizationparameter calculating unit performs determination of selectioncandidates in an order set beforehand, and sets the predictionquantization parameter based on the determination result.

(4) The image decoding device according to (1), wherein the quantizationparameter calculating unit selects, based on determination informationincluded in the stream information, performing one or the other ofprocessing of setting to the prediction quantization parameter aquantization parameter in an order indicated by identificationinformation included in the stream information, and processing ofdetermining selection candidates in an order set beforehand and settingthe prediction quantization parameter based on the determination result.

(5) The image decoding device according to any one of (1) through (4),wherein the quantization parameter calculating unit takes the selectioncandidates, having excluded from the adjacent decoded blocks at leastblocks where quantization parameters are redundant or blocks whereinverse quantization using quantization parameters is not performed.

(6) The image decoding device according to any one of (1) through (5),wherein, in the event that there is no selection candidate, thequantization parameter calculating unit takes a quantization parameterof an initial value in a slice as the prediction quantization parameter.

(7) The image decoding device according to any one of (1) through (6),wherein the quantization parameter calculating unit includes aquantization parameter updated last in the selection candidates.

(8) The image decoding device according to any one of (1) through (7),wherein the quantization parameter calculating unit calculates aquantization parameter of the block to be decoded by adding differencewhich the difference information indicates to the predictionquantization parameter.

(9) An image encoding device, comprising:

a control unit configured to set a quantization parameter as to a blockto be encoded;

an information generating unit configured to take quantizationparameters of encoded blocks spatially or temporally adjacent to a blockto be encoded, as selection candidates, select from the selectioncandidates a prediction quantization parameter in accordance to the setquantization parameter, and generate difference information indicatingdifference between the prediction quantization parameter and the setquantization parameters; and

an encoding unit configured to include the difference information instream information generated by performing encoding processing of theblock to be encoded, using the set quantization parameter.

(10) The image encoding device according to (9), wherein the informationgenerating unit selects a quantization parameter of which the differenceas to the set quantization parameter is the smallest, as the predictionquantization parameter.

(11) The image encoding device according to (10), wherein theinformation generating unit generates identification informationindicating the order of blocks as to the selected quantizationparameter, with the adjacent encoded blocks in a predetermined order;

and wherein the encoding unit includes the identification information inthe stream information.

(12) The image encoding device according to (11), wherein theinformation generating unit takes an order of array where priority isgiven to one of an encoded block adjacent to the left side, an encodedblock adjacent above, and an encoded block temporally adjacent.

(13) The image encoding device according to either of (11) or (12),wherein the information generating unit can switch the order of array ofadjacent encoded blocks.

(14) The image encoding device according to (9), wherein the informationgenerating unit performs determination of selection candidates in anorder set beforehand, and selects the prediction quantization parameterbased on the determination result.

(15) The image encoding device according to (9), wherein the informationgenerating unit is capable of selecting between processing of selectinga quantization parameter of which the difference as to the setquantization parameter is the smallest as the prediction quantizationparameter, and processing of performing determination of selectioncandidates in an order set beforehand and selecting the predictionquantization parameter based on the determination result, and generatesdetermination information indicating the selected processing;

and wherein the encoding unit includes the determination information inthe stream information.

(16) The image encoding device according to any one of (9) through (15),wherein the information generating unit takes the selection candidates,having excluded from the adjacent encoded blocks at least blocks wherequantization parameters are redundant or blocks where quantization usingquantization parameters is not performed.

(17) The image encoding device according to any one of (9) through (16),wherein, in the event that there is no selection candidate, theinformation generating unit generates difference information indicatingdifference between a quantization parameter of an initial value in aslice, and the set quantization parameters.

(18) The image encoding device according to any one of (9) through (17),wherein the information generating unit includes a quantizationparameter updated last in the selection candidates.

INDUSTRIAL APPLICABILITY

With the image decoding device, image encoding device, and methodthereof, according to this technology, quantization parameters ofencoded blocks spatially or temporally adjacent to a block to be encodedare taken as selection candidates, and a prediction quantizationparameter is selected from the selection candidates in accordance to theset quantization parameter. Difference information indicating differencebetween the prediction quantization parameter and the quantizationparameters set as to the block to be encoded is generated. Accordingly,the difference of quantization parameters can be prevented from becominga great value, and encoding efficiency of quantization parameters can beimproved. Also, in a case of decoding stream information wheredifference information is included, a prediction quantization parameteris selected from quantization parameters of decoded blocks spatially ortemporally adjacent to a block to be decoded, and a quantizationparameter of the block to be decoded is calculated from the predictionquantization parameter and the difference information. Accordingly, evenin the event that stream information is generated with improved encodingefficiency of quantization parameters, in the event of decoding thisstream information the quantization parameters can be restored based onthe prediction quantization parameter and difference information, anddecoding processing can be correctly performed. Accordingly, this issuitable for equipment which transmits/receives stream informationobtained by performing encoding in block increments, via network mediasuch as satellite broadcasting, cable TV, the Internet, cellulartelephones, and the like, and equipment and the like which processesthis on storage media such as optical discs, magnetic disks, flashmemory, and so forth.

REFERENCE SIGNS LIST

-   -   10 image encoding device    -   11 A/D conversion unit    -   12, 57 screen rearranging buffer    -   13 subtracting unit    -   14 orthogonal transform unit    -   15 quantization unit    -   16 lossless encoding unit    -   17, 51 storage buffer    -   18 rate control unit    -   19 information generating unit    -   21, 53 inverse quantization unit    -   22, 54 inverse orthogonal transform unit    -   23, 55 adding unit    -   24, 56 deblocking filter    -   26, 61 frame memory    -   31, 71 intra prediction unit    -   32 motion prediction/compensation unit    -   33 prediction image/optimal mode selecting unit    -   50 image decoding device    -   52 lossless decoding unit    -   58 D/A converting unit    -   59 quantization parameter calculating unit    -   62, 73 selector    -   72 motion compensation unit    -   80 computer device    -   90 television apparatus    -   92 cellular telephone    -   94 recording/playback device    -   96 imaging apparatus    -   191 quantization parameter memory unit    -   192 difference computing unit    -   591 computing unit    -   592 quantization parameter memory unit

1. An image encoding apparatus comprising: circuitry configured to:calculate, if a block adjacent to the left of a to-be-encoded blockcannot be referenced and a block adjacent above the to-be-encoded blockcan be referenced, a prediction quantization parameter of theto-be-encoded block using a quantization parameter of the block adjacentabove; calculate, if the block adjacent to the left can be referencedand the block adjacent above cannot be referenced, the predictionquantization parameter of the to-be-encoded block using a quantizationparameter of the block adjacent to the left; generate differenceinformation indicating a difference between the prediction quantizationparameter and a quantization parameter set for the to-be-encoded block;and generate a bit stream comprising the difference information.
 2. Theimage encoding apparatus according to claim 1, wherein the block is acoding unit, and the coding unit has a hierarchical structure.
 3. Theimage encoding apparatus according to claim 2, wherein the circuitrygenerates the bit stream by performing encoding processing using thequantization parameter set for the to-be-encoded block.
 4. The imageencoding apparatus according to claim 3, wherein the circuitry generatesdifference information indicating a difference between an initial valueof a quantization parameter for a slice and the set quantizationparameter, in a case that no block can be referenced.
 5. The imageencoding apparatus according to claim 1, wherein the circuitry generatesthe bit stream by performing encoding processing using the quantizationparameter set for the to-be-encoded block.
 6. The image encodingapparatus according to claim 5, wherein the circuitry generatesdifference information indicating a difference between an initial valueof a quantization parameter for a slice and the set quantizationparameter, in a case that no block can be referenced.
 7. The imageencoding apparatus according to claim 1, wherein the circuitry generatesdifference information indicating a difference between an initial valueof a quantization parameter for a slice and the set quantizationparameter, in a case that no block can be referenced.
 8. An imageencoding method comprising: calculating, if a block adjacent to the leftof a to-be-encoded block cannot be referenced and a block adjacent abovethe to-be-encoded block can be referenced, a prediction quantizationparameter of the to-be-encoded block using a quantization parameter ofthe block adjacent above; calculating, if the block adjacent to the leftcan be referenced and the block adjacent above cannot be referenced, theprediction quantization parameter of the to-be-encoded block using aquantization parameter of the block adjacent to the left; generatingdifference information indicating a difference between the predictionquantization parameter and a quantization parameter set for theto-be-encoded block; and generating a bit stream comprising thedifference information.
 9. A non-transitory computer-readable mediumhaving embodied thereon a program, which when executed by a computercauses the computer to execute a method, the method comprising:calculating, if a block adjacent to the left of a to-be-encoded blockcannot be referenced and a block adjacent above the to-be-encoded blockcan be referenced, a prediction quantization parameter of theto-be-encoded block using a quantization parameter of the block adjacentabove; calculating, if the block adjacent to the left can be referencedand the block adjacent above cannot be referenced, the predictionquantization parameter of the to-be-encoded block using a quantizationparameter of the block adjacent to the left; generating differenceinformation indicating a difference between the prediction quantizationparameter and a quantization parameter set for the to-be-encoded block;and generating a bit stream comprising the difference information. 10.An image decoding apparatus comprising: circuitry configured to: obtain,when a block adjacent to the left of a current block cannot bereferenced and a block adjacent above the current block can bereferenced, a prediction quantization parameter derived from aquantization parameter of the block adjacent above; obtain, when theblock adjacent to the left can be referenced and the block adjacentabove cannot be referenced, the prediction quantization parameterderived from a quantization parameter of the block adjacent to the left;obtain, from bit stream information, a difference information indicatinga difference between the obtained prediction quantization parameter anda quantization parameter of the current block; and generate thequantization parameter of the current block, from the predictionquantization parameter and the difference information.
 11. The imagedecoding apparatus according to claim 10, wherein the block is a codingunit, and the coding unit has a hierarchical structure.
 12. The imagedecoding apparatus according to claim 11, wherein the circuitry decodesthe bit stream by performing decoding processing.
 13. The image decodingapparatus according to claim 12, wherein the circuitry obtainsdifference information indicating a difference between an initial valueof a quantization parameter for a slice and the quantization parameterof the current block, in a case that no block can be referenced.
 14. Theimage decoding apparatus according to claim 10, wherein the circuitrydecodes the bit stream by performing decoding processing.
 15. The imagedecoding apparatus according to claim 14, wherein the circuitry obtainsdifference information indicating a difference between an initial valueof a quantization parameter for a slice and the quantization parameterof the current block, in a case that no block can be referenced.
 16. Theimage decoding apparatus according to claim 10, wherein the circuitryobtains difference information indicating a difference between aninitial value of a quantization parameter for a slice and thequantization parameter of the current block, in a case that no block canbe referenced.
 17. An image decoding method comprising: obtaining, whena block adjacent to the left of a current block cannot be referenced anda block adjacent above the current block can be referenced, a predictionquantization parameter derived from a quantization parameter of theblock adjacent above; obtaining, when the block adjacent to the left canbe referenced and the block adjacent above cannot be referenced, theprediction quantization parameter derived from a quantization parameterof the block adjacent to the left; obtaining, from bit streaminformation, a difference information indicating a difference betweenthe obtained prediction quantization parameter and a quantizationparameter of the current block; and generating the quantizationparameter of the current block, from the prediction quantizationparameter and the difference information.
 18. The image decoding methodaccording to claim 17, wherein the block is a coding unit, and thecoding unit has a hierarchical structure.
 19. The image decoding methodaccording to claim 18, further comprising decoding the bit stream byperforming decoding processing.
 20. The image decoding method accordingto claim 19, further comprising obtaining difference informationindicating a difference between an initial value of a quantizationparameter for a slice and the quantization parameter of the currentblock, in a case that no block can be referenced.
 21. The image decodingmethod according to claim 17, further comprising decoding the bit streamby performing decoding processing.
 22. The image decoding methodaccording to claim 21, further comprising obtaining differenceinformation indicating a difference between an initial value of aquantization parameter for a slice and the quantization parameter of thecurrent block, in a case that no block can be referenced.
 23. The imagedecoding method according to claim 17, further comprising obtainingdifference information indicating a difference between an initial valueof a quantization parameter for a slice and the quantization parameterof the current block, in a case that no block can be referenced.
 24. Anon-transitory computer-readable medium having embodied thereon aprogram, which when executed by a computer causes the computer toexecute a method, the method comprising: obtaining, when a blockadjacent to the left of a current block cannot be referenced and a blockadjacent above the current block can be referenced, a predictionquantization parameter derived from a quantization parameter of theblock adjacent above; obtaining, when the block adjacent to the left canbe referenced and the block adjacent above cannot be referenced, theprediction quantization parameter derived from a quantization parameterof the block adjacent to the left; obtaining, from bit streaminformation, a difference information indicating a difference betweenthe obtained prediction quantization parameter and a quantizationparameter of the current block; and generating the quantizationparameter of the current block, from the prediction quantizationparameter and the difference information.